# A Piecewise Linear Regression Model Ensemble for Large-Scale Curve Fitting
## 6 Empirical Results
### Comparative Analysis of the Proposed Methods  
As summarised in Figure 12, all the tested combination approaches behave in the same way when increasing \(n\) and \(M\): they improve in terms of bias and variance. The model selection, as expected, does not take advantage of increasing values of \(M\) to reduce variance, and it only improves with increasing \(n\) in the same way that the reference algorithm did.  
This section aims at comparing the performance of the proposed methods. In particular, Figures 17 and 18 present the error and complexity curves of the methods for a small and a large sub-sample size. Both figures clearly show that MS is always the worst option because it produces models with the highest generalisation error. Note that the complexity of these models is always lower than the \(K\) chosen by the reference algorithm because of the fact that BIC selects one model among under-parameterised LHMs.  
Regarding the combination methods, it can be seen that when a small value of \(n\) is chosen, the MSE(TTS) stabilises for \(M\geq 50\). Once the variance cannot be reduced much more by increasing \(M\), almost all the MSE is due to the bias of the reference algorithm for that \(n\). In the case of \(n=1\) M, for both problems, the error curves seem not to be stabilised yet, so combining a larger number of models could reduce the MSE in terms of variance even more. In the particular case of MC1, it can be seen that in terms of error, this method provides very low values of MSE due to its lack of compression, which allows its final model to have all the richness of the combined sub-models. Note that the complexity of this method has not been plotted because it is always fixed to 10 k.  
Figure 16: Bias–variance decomposition of the MSE(TTS) of the proposed methods in problem PR4. For each method: **Top:** Bias and variance varying \(n\), with fixed \(M=50\). **Bottom:** Bias and variance varying \(M\), with fixed \(n=500\) k. The background area represents the decomposition of the MSE(TTS) of the reference algorithm for this problem and a training dataset of \(N=10\) M.  
Concerning method MC2, it can be observed that its \(K\) systematically increases with \(M\) (similar to the reference algorithm with increasing \(N\)), and its results in terms of error are consistent with the reference algorithm.  
Figure 17: Comparison of the performance of the proposed methods in PR1, varying the number of sub-samples \(M\) (distributions calculated from 30 replications). **Left:** Small sub-sample size (\(n=1\) k). **Right:** Big sub-sample size (\(n=1\) M). **Top:** Distribution of the MSE over the TTS. **Bottom:** Distribution of the number of knots (\(K\)). The black dashed lines represent the 25 th and 75 th percentiles of the results obtained with the reference algorithm and a sample size of \(N=10\) M.  
Figure 18: Comparison of the performance of the proposed methods in PR4, varying the number of sub-samples \(M\) (distributions calculated from 30 replications). **Left:** Small sub-sample size (\(n=1\) k). **Right:** Big sub-sample size (\(n=1\) M). **Top:** Distribution of the MSE over the TTS. **Bottom:** Distribution of the number of knots (\(K\)). The black dashed lines represent the 25 th and 75 th percentiles of the results obtained with the reference algorithm and a sample size of \(N=10\) M.  
more and more competitive at the same time. On the other hand, MC3 always fixes a quite stable complexity (larger than the \(K\) estimated by the reference algorithm) for any value of \(M\), and in terms of error seems to be the best option in every case. Note that despite having many fewer knots than the models obtained with MC1, MC3 always produces equal or better results. Furthermore, comparing MC2 and MC3, it can be seen that for high values of \(M\) both have similar results in terms of MSE.