## 4 Experiments and Analysis
### Experimental Results and Analysis
#### 4.4.1 Forecasting Accuracy and Performance Gap  
To evaluate the prediction quality of the proposed meta-learning-based model selection framework, two complementary metrics were examined: the absolute forecasting accuracy, measured using the root mean squared error (RMSE), and the relative performance gap from the optimal model selected via exhaustive search. Together, these metrics provide a comprehensive view of how accurately and efficiently the system operates under varying computational resource conditions and forecasting demands. Figure 4 presents the RMSE values of the proposed method compared to the exhaustive search baseline across five CPU levels and varying numbers of concurrently served sensors. In single-sensor scenarios, the proposed method achieved a nearly identical RMSE to that of the exhaustive approach, regardless of CPU availability. Even as the number of sensors increased to 100 under 20% CPU availability--a highly constrained condition--the increase in RMSE remained modest, typically under 0.03. This indicates that the system maintains stable and reliable forecasting performance even when operating with limited resources and high service concurrency. The performance of the proposal increased when the CPU level is above 40%. It is because as the available resources increase, more accurate models can be selected. On the other hand, the performance shows few changes after 60% of CPU since the computing resources can afford to run the best-performing model. While the RMSE reveals the absolute accuracy, it does not capture how close the selected model is to the best possible choice under each condition. To address this, the performance gap was measured, defined as the relative increase in prediction error compared to the optimal model identified via exhaustive search:  
\[\text{Performance Gap}=\sum_{i\in\text{metrics}}w_{i}\cdot\left(\frac{S_{i}-O_{i }}{O_{i}}\right) \tag{10}\]  
where \(S_{i}\) and \(O_{i}\) represent the error values (e.g., RMSE or MAE) of the selected and optimal models, respectively, and \(w_{i}\) is a weight applied to each metric. Figure 4: Comparison of the forecasting accuracy (RMSE) between the proposed meta-learning method and exhaustive search across different CPU availability levels (20–100%) and sensor concurrency scenarios (1, 10, 50, and 100 sensors). As shown in Figure 5, the performance gap remained negligible for single-sensor cases (close to 0%) and peaked at around 12.4% under 20% CPU resource availability with 10 concurrent sensors. The gap decreased again when the concurrency increased to 50 and 100 sensors, indicating stable generalization through fallback to simpler models. This indicates that the proposal maintains stable and reliable forecasting performance even when operating with limited resources and high service concurrency. The results suggest that although the system selects models under strict resource constraints, it consistently chooses models that are close to optimal in terms of forecasting performance. The slight increase in the performance gap under heavy sensor loads can be attributed to the system's resource-aware selection logic. When faced with limited computing capacity, the meta-learner prioritizes inference efficiency, often favoring simpler models over more accurate but computationally intensive ones. Despite this trade-off, the overall accuracy loss remains minor, validating the framework's ability to intelligently balance accuracy and resource efficiency in real time. As the available CPU resources increase to 40% or higher, especially when the number of concurrent models reaches 10, the performance gap shows a dramatic reduction. This phenomenon can be explained by the system's increased capacity to handle computational demands. With more resources, the framework is able to select and execute complex, high-performing models such as TFT, which were previously substituted with simpler models under stricter resource constraints.