# Integrating Interpolation and Extrapolation: A Hybrid Predictive Framework for Supervised Learning
## 3 Numerical Experiments
### Results and Discussion  
The experiments are conducted on a computer with an AMD Ryzen 5 4600U and 16 GB (3200 MHz) RAM under the Windows 10 operating system. The models are implemented in Python programming language using Gurobi 9.5.2 as the solver. The optimal hyperparameter \(k^{*}\) for the \(k\)NN model and \(h^{*}\) for our hybrid prediction framework are determined for each data batch, according to the procedures described in Section 3.3. The \(k^{*}\) and \(h^{*}\) for each batch are shown in Tables 4 and 5, respectively. For an insightful comparative analysis, we examine the MSEs of three models M1 (kNN), M2 (linear regression), and M3 (hybrid prediction model) on the test dataset of every data batch, focusing on the predicted ship deficiency numbers, and the results are shown in Table 6 and Figure 4. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Batch** & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** & **9** & **10** \\ \hline \(k^{*}\) & 2 & 1 & 6 & 1 & 2 & 5 & 6 & 8 & 10 & 5 \\ \hline
**Batch** & **11** & **12** & **13** & **14** & **15** & **16** & **17** & **18** & **19** & **20** \\ \hline
**Batch** & **21** & **12** & **13** & **14** & **15** & **16** & **17** & **18** & **19** & **20** \\ \hline \(k^{*}\) & 5 & 5 & 6 & 7 & 7 & 7 & 3 & 4 & 5 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Optimal hyperparameter \(k^{*}\) for 30 data batches. \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline
**Batch** & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** & **9** & **10** \\ \hline \(h^{*}\) & 0.4 & 0.1 & 1 & 2.5 & 3.3 & 2 & 0.1 & 1.5 & 0.8 & 0.1 \\ \hline
**Batch** & **11** & **12** & **13** & **14** & **15** & **16** & **17** & **18** & **19** & **20** \\ \hline \(h^{*}\) & 4.7 & 5.7 & 2.7 & 2 & 3 & 1.2 & 3.3 & 0.5 & 1.3 & 1.7 \\ \hline
**Batch** & **21** & **22** & **23** & **24** & **25** & **26** & **27** & **28** & **29** & **30** \\ \hline \(h^{*}\) & 2.1 & 1.9 & 2.1 & 3.2 & 2.7 & 1 & 0.3 & 0.1 & 0.8 & 0.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Optimal hyperparameter \(h^{*}\) for 30 data batches. These results are then categorized into three distinct groups: data batches with the minimum, intermediate, or maximum MSEs in model M3 (hybrid prediction model), which are illustrated in Figures 5-7, respectively. The results of computational experiments demonstrate that M3 (the hybrid prediction model) consistently surpasses M1 (\(k\)NN) or M2 (linear regression) in a majority of the data batches, yielding lower MSEs, which validates the effectiveness of our proposed hybrid predictive framework. Specifically, M3 excels over M1 or M2 in 86.7% of the 30 data batches. \begin{table}
\begin{tabular}{c c c c c c c c c c c} \hline \hline
**Batch** & **1** & **2** & **3** & **4** & **5** & **6** & **7** & **8** & **9** & **10** \\ \hline M1 (\(k\)NN) & 4.75 & 3.60 & 2.13 & 19.10 & 18.71 & 15.08 & 3.12 & 42.87 & 20.31 & 8.16 \\ M2 (LR) & 4.69 & 12.00 & 30.97 & 16.91 & 19.49 & 9.29 & 3.62 & 38.89 & 34.20 & 50.52 \\ M3 (hybrid) & 4.42 & 11.89 & 29.77 & 19.92 & 18.50 & 10.44 & 3.82 & 40.88 & 22.52 & 49.73 \\ \hline
**Batch** & **11** & **12** & **13** & **14** & **15** & **16** & **17** & **18** & **19** & **20** \\ \hline M1 (\(k\)NN) & 17.38 & 20.90 & 11.34 & 8.53 & 12.84 & 8.20 & 10.61 & 13.17 & 9.84 & 5.95 \\ M2 (LR) & 30.68 & 28.03 & 10.71 & 32.78 & 11.65 & 8.12 & 11.84 & 19.58 & 20.88 & 9.29 \\ M3 (hybrid) & 17.14 & 22.81 & 10.36 & 14.61 & 12.55 & 7.23 & 10.19 & 19.40 & 20.63 & 5.63 \\ \hline
**Batch** & **21** & **22** & **23** & **24** & **25** & **26** & **27** & **28** & **29** & **30** \\ \hline M1 (\(k\)NN) & 10.37 & 31.92 & 7.83 & 5.23 & 19.05 & 13.40 & 6.47 & 14.82 & 14.73 & 21.60 \\ M2 (LR) & 22.30 & 27.94 & 5.98 & 89.45 & 21.63 & 15.81 & 25.48 & 10.55 & 74.54 & 20.06 \\ M3 (hybrid) & 19.65 & 28.94 & 7.71 & 60.83 & 18.64 & 18.98 & 24.89 & 11.66 & 74.59 & 20.36 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: MSEs of three models M1, M2, and M3 for 30 data batches.