## 4 Proposed Inference Latency Prediction Approaches
### Proposed Inference Latency Prediction between Object Detection Algorithms  
This section describes the second method (Method 2), wherein the client with the SSD-MobileNet model predicts the inference latency of the RFCN model for the client, edge, and cloud. It is very efficient to predict the inference latency of the RFCN model through the SSD-MobileNet model because the SSD-MobileNet model operates much faster than the RFCN model, as explained in Section 3.2. To analyze the relationship between the inference latencies of the RFCN and SSD-MobileNet models, we compare the correlation coefficients \(\rho\) between the normalized inference latency of the SSD-MobileNet model for the client, \(Z_{client}^{SSD-MobileNet}\), and the normalized inference latency of the RFCN model for the client, edge, and cloud, i.e., \(Z_{client}^{RFCN}\), \(Z_{edge}^{RFCN}\), and \(Z_{cloud}^{RFCN}\), as listed in Table 5. The two protocols show somewhat different characteristics. In gRPC, \(Z_{client}^{SSD-MobileNet}\) is observed to have a negative correlation with \(Z_{client}^{RFCN}\), \(Z_{edge}^{RFCN}\), and \(Z_{cloud}^{RFCN}\), and in REST, \(Z_{client}^{SSD-MobileNet}\) is observed to have a negative correlation with \(Z_{client}^{RFCN}\) and \(Z_{edge}^{RFCN}\) and a positive correlation with \(Z_{cloud}^{RFCN}\). Therefore, it is possible to predict the inference latency of the RFCN model through that of the SSD-MobileNet model by exploiting these correlation characteristics. We compared the relationship between the normalized inference latencies of the SSD-MobileNet and RFCN models by considering a factor, \(\alpha\), to reflect the negative correlation. For the positive and negative correlations, \(\alpha\) was set to +1 and \(-1\), respectively. Then, the normalized inference latencies of the RFCN model for the client, edge, and cloud, \(Z_{client}^{RFCN}\), \(Z_{edge}^{RFCN}\), and \(Z_{cloud}^{RFCN}\), are predicted as \(\alpha\cdot Z_{client}^{SSD-MobileNet}\). Therefore, with the gRPC protocol, the normalized inference latencies of the RFCN model for the client, edge, and cloud are predicted as \(-Z_{client}^{SSD-MobileNet}\), \(-Z_{client}^{SSD-MobileNet}\), and \(Z_{client}^{SSD-MobileNet}\), respectively. Figure 10 shows the normalized inference latencies of the RFCN and SSD-MobileNet models for the client in the REST protocol. It can be observed that the normalized inference latencies of the client in the RFCN and SSD-MobileNet models have a negative correlation as shown in Figure 10a. By setting \(\alpha\) as -1, we can observe that the normalized inference latencies of the client tend to be similar for the RFCN and SSD-MobileNet models, as shown in Figure 10b. Figure 11 shows the normalized inference latencies of the RFCN model for the edge device and the SSD-MobileNet model for the client device in the REST protocol. Similar to what is observed in the relationship between the RFCN and SSD-MobileNet models for the client device, it can be observed that the normalized inference latencies of the RFCN model for the edge device and the SSD-MobileNet model for the client device have a negative correlation, as shown in Figure 11a. Therefore, it is possible to approximately predict the normalized inference latency of the RFCN model for the edge device by multiplying the normalized inference latency of the SSD-MobileNet model for the client by \(-1\), as shown in Figure 11b. Similarly, the normalized inference latency of the RFCN for the cloud device \begin{table}
\begin{tabular}{c c c c} \hline \hline \(\rho\) & **Client** & **Edge** & **Cloud** \\ \hline gRPC & –0.39 & –0.36 & –0.26 \\ REST & –0.39 & –0.32 & 0.38 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The correlation coefficients of normalized inference latency between the SSD-MobileNet model for the client and the RFCN model for the client, edge, and cloud. can be predicted based on the normalized inference latency of the SSD-MobileNet model for the client device.