# A Piecewise Linear Regression Model Ensemble for Large-Scale Curve Fitting
## 5 Proposed Methods
### Model Combination (MC)
#### 5.2.2 Model Averaging with Complexity Reduction and Refitting (MC2)
```
PART I: Generate the pruning sequence (a) initialisation \(\mathbf{k}=\mathbf{k}^{I}\), \(\mathbf{h}=\mathbf{h}^{I}\) \(PM_{m=K^{I}}=\{\mathbf{k},\mathbf{h}\}\), \(K_{m=K^{I}}=K^{I}\) for\(\mathbf{m}=(K^{I}-1)\) down to \(K_{min}\)do (b) calculate the \(p\)-value of each knot for\(j=2\) to \((K_{m}-1)\)do\(\triangleright\) for all the internal knots \(h_{j}^{*}=\frac{k_{j}-k_{j-1}}{k_{j+1}-k_{j-1}}\cdot(h_{j+1}-h_{j-1})+h_{j-1}\) \(p_{j}=p\)-value\((h_{j}^{*})\) with \(N(h_{j},se(h_{j}))\) endfor (c) prune the knot with the highest \(p\)-value \(\mathbf{k}=\mathbf{k}-\{k_{d}\}\), \(\mathbf{h}=\mathbf{h}-\{h_{d}\}\) where \(d=\underset{j}{\arg\max}\ p_{j}\) \(PM_{m}=\{\mathbf{k},\mathbf{h}\}\) (d) save the \(p\)-value of the pruned knot \((p_{m})\) and \(K_{m}\) \(K_{m}=K_{m+1}-1\), \(p_{m}=p_{d}\) (e) calculate the slope of the \(p\)-value curve in m \(sl_{m}=\frac{p_{m+1}-p_{m}}{K_{m+1}-K_{m}}\) endfor  PART II: Select the final PM and \(K\) (a) calculate the maximum slope that is allowed \(sl_{max}=\frac{0.05-p_{K_{min}}}{K_{0.05}-K_{min}}\) (b) find the elbow of the \(p\)-value curve \(f=sl>sl_{max}\) (c) select the final \(K\) and \(PM\) \(K=K_{f(1)},PM=PM_{f(1)}\)
```  
**Algorithm 1** Pruning algorithm of MC2. Given the maximum complexity (\(K^{l}\)) and fixing the minimum one to the lowest \(K_{s}\) of the sub-models (\(K_{min}=min(K_{1},K_{2},...,K_{M})\)), the pruning sequence of the models is built by pruning the knots one by one in that way. In each iteration \(m\), the pruned model (PM\({}_{m}\)) is saved, as well as its complexity \(K_{m}\), and the \(p\)-value of the last knot that has been pruned (\(p_{m}\)). Therefore, once the pruning procedure has finished, a set of \(K^{l}-K_{min}\) models is obtained, each one associated with the \(p\)-value of the knot that has been pruned in its iteration. Figure 5 shows the \(p\)-value sequence for each PM after the whole pruning process. It can be observed that, as expected, the \(p\)-value of the knots decreases during the process, pruning from the most irrelevant knots with \(p\)-values near \(1\) to some of them with \(p\)-values very close to \(0\). Once the pruning sequence has been built, the final complexity \(K\) is selected by looking for the elbow of the \(p\)-value curve (see Figure 5). Specifically, the last point of the curve with a lower slope than the one given by the line that connects the end point of the pruning  
Figure 4: Example of one iteration of the pruning algorithm: the \(p\)-value associated with each knot is calculated in this way, and the one with the highest value is removed. In this example, where only the standard error of two knots is plotted for simplicity, the knot \(j\) would be pruned. Figure 5: Example of a complete pruning sequence of the method MC2. The zoom in the interest area shows how the elbow of the curve has been determined, and the selected \(K\) value. sequence and the point of \(p\)-value = 0.05 is chosen. The selected model after the pruning step is the PM of the sequence with that value of \(K\). Finally, a refitting step is carried out to improve the knots' positions. For this, a new cloud of points is created, evaluating the sub-models in several x-coordinates. Constrained by the available memory, the number of points for this final step (\(Q\)) is fixed. In this paper, \(Q\) is fixed to 500 k. Then, each sub-model is evaluated in a random set of x-coordinates of size \(Q/M\). These points are uniformly generated over the x-range to avoid repetition in different x-coordinates as much as possible. Once this new dataset is generated, the refitting step of the original LHM algorithm is executed (its details can be seen in [9]), fixing the final position of the knots of the model.