## 3 Results
The table also includes each metric's macro average (averaging the unweighted mean per label), the weighted average (averaging the support-weighted mean per class), and the precision each model achieves. Figure 4: PR curves for the wave overtopping models using the testing dataset. \begin{table}
\begin{tabular}{r r|r r|r r|r r} \hline \hline  & \multicolumn{4}{c|}{**Predicted**} \\ \cline{2-9}  & \multicolumn{1}{c|}{**Threshold**} & \multicolumn{2}{c|}{**0.25**} & \multicolumn{2}{c|}{**0.5**} & \multicolumn{2}{c}{**0.75**} \\ \hline
**Model** & **Real Class** & **No Overtop.** & **Overtop.** & **No Overtop.** & **Overtop.** & **No Overtop.** & **Overtop.** \\ \hline \multirow{2}{*}{1} & No overtop. & 2162 & 57 & 2217 & 2 & 2219 & 0 \\  & Overtop. & 1 & 26 & 19 & 8 & 27 & 0 \\ \hline \multirow{2}{*}{2} & No overtop. & 2191 & 28 & 2219 & 0 & 2219 & 0 \\  & Overtop. & 4 & 23 & 27 & 0 & 27 & 0 \\ \hline \multirow{2}{*}{3} & No overtop. & 3670 & 40 & 3708 & 2 & 3709 & 1 \\  & Overtop. & 8 & 47 & 15 & 40 & 29 & 26 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Confusion matrices for the three wave overtopping models using a 0.25, 0.5, and 0.75 threshold decision boundary on the testing data predictions. Table 6 confirms the data in Table 5: models 1 and 2 detect more overtopping events on the testing data using the 0.25 threshold (they have a higher recall). For higher thresholds, model 3 outperforms models 1 and 2. Model 3 has a higher precision and F1 score than models 1 and 2 at any threshold. As previously explained, once deployed in the production system, the wave overtopping models will input the Portus forecast data to make an overtopping prediction. Model 1, trained on real historical data, uses the same input variables as model 3, trained on historical data of the Portus forecast system. The previous test results were obtained using the corresponding test dataset created from the original dataset. I.e., model 1 was tested with real historical data and model 3 with historical forecasted data. Now, we will test both model 1 and model 3 on historical forecasted data and compare them to see how well model 1 will behave using forecasted data like it would in the production system. Figure 5 shows the PR curves for models 1 and 3 using the testing data of model 3 (i.e., using historical forecasted data). \begin{table}
\begin{tabular}{c c|c c c|c c c|c c c|c c c|c} \hline \hline  & & \multicolumn{3}{c|}{**Precision**} & \multicolumn{3}{c|}{**Recall**} & \multicolumn{3}{c|}{**F1**} & \multicolumn{3}{c|}{**Accuracy**} & \\ \hline
**Model** & **Threshold** & **0.25** & **0.50** & **0.75** & **0.25** & **0.50** & **0.75** & **0.25** & **0.50** & **0.75** & **0.25** & **0.50** & **0.75** & **Support** \\ \hline \multirow{4}{*}{1} & No overtop. & 1.00 & 0.99 & 0.99 & 0.97 & 1.00 & 1.00 & 0.99 & 1.00 & 0.99 & & & & 2219 \\  & Overtop. & 0.31 & 0.80 & 0.00 & 0.96 & 0.30 & 0.00 & 0.47 & 0.43 & 0.00 & & & & 27 \\  & Macro avg & 0.66 & 0.90 & 0.49 & 0.97 & 0.65 & 0.50 & 0.73 & 0.71 & 0.50 & 0.97 & 0.99 & 0.66 & 2246 \\  & Weigh. avg & 0.99 & 0.99 & 0.98 & 0.97 & 0.99 & 0.99 & 0.98 & 0.99 & 0.98 & & & & 22246 \\ \hline \multirow{4}{*}{2} & No overtop. & 1.00 & 0.99 & 0.99 & 0.99 & 1.00 & 1.00 & 0.99 & 0.99 & 0.99 & & & & 2219 \\  & Overtop. & 0.45 & 0.00 & 0.00 & 0.85 & 0.00 & 0.00 & 0.59 & 0.00 & 0.00 & & & 27 \\  & Macro avg & 0.72 & 0.49 & 0.49 & 0.92 & 0.50 & 0.50 & 0.79 & 0.50 & 0.50 & 0.99 & 0.99 & 0.72 & 2246 \\  & Weigh. avg & 0.99 & 0.98 & 0.98 & 0.99 & 0.99 & 0.99 & 0.99 & 0.98 & 0.98 & & & 2246 \\ \hline \multirow{4}{*}{3} & No overtop. & 1.00 & 1.00 & 0.99 & 0.99 & 1.00 & 1.00 & 0.99 & 1.00 & 1.00 & & & 3710 \\  & Overtop. & 0.54 & 0.95 & 0.96 & 0.85 & 0.73 & 0.47 & 0.66 & 0.82 & 0.63 & & & 55 \\ \cline{1-1}  & Macro avg & 0.77 & 0.97 & 0.98 & 0.92 & 0.86 & 0.74 & 0.83 & 0.91 & 0.82 & 0.99 & 1.00 & 0.77 & 3765 \\ \cline{1-1}  & Weigh. avg & 0.99 & 1.00 & 0.99 & 0.99 & 1.00 & 0.99 & 1.00 & 0.99 & & & & 3765 \\ \hline \hline \end{tabular}
\end{table}