# A Least Squares Ensemble Model Based on Regularization and Augmentation Strategy
## 1 Introduction  
Computational simulations, such as finite element analysis (FEA) or computational fluid dynamics, have been displaying steady progress in describing engineering systems, and these simulations play a key role in optimizing the design of complex engineering equipment. However, computer simulations may consume a considerable amount of time for complicated simulations in engineering design. Therefore, a surrogate modeling method has been developed rapidly over the last three decades as an alternative for computationally expensive simulations that consumes less time [1]. A wide variety of surrogate models have been used in engineering design, such as polynomial response surface (PRS) [2; 3], Kriging (KRG) [3; 4; 5; 6], radial basis function (RBF) [7; 8], and support vector regression (SVR) [8; 9; 10; 11; 12; 13]. The PRS and SVR models can identify global trends for a given input data set; whereas, owing to the interpolation characteristics, KRG and RBF have higher local accuracy around the training points. Reviews about the surrogate models can be found in [14; 15; 16; 17]. The rapid development of various surrogate models provides researchers with a lot more flexibility while they are selecting models for engineering design problems. However, it is challenging to choose the optimal model for a specific application before all the different surrogate models are constructed [18]. Since practical engineering applications often exhibit different linear and nonlinear properties, no single surrogate model can exhibit high performance in all scenarios; each surrogate model has advantages and disadvantages [19; 20; 21; 22]. The PRS model fits the linear relationship between the inputs and the outputs well, while the KRG and RBF model are more suited for complex nonlinear relationships between the input and output datasets. The SVR model is suitable for both linear and nonlinear relationships between the inputs and the outputs, as it can choose different kernel functions and hyper-parameters [23; 24; 25; 26; 27; 28; 29; 30]. To make better use of the advantages of each model, as well as avoid wasting existing individual models, researchers have combined different surrogate models into an ensemble model to develop weighted average surrogate (WAS) models [31; 32]. Several ensemble models have been developed in the literature, and studies have shown that the ensemble model combines the predictive power of each individual surrogate model to improve accuracy and robustness. Existing ensemble models are most commonly based on error correlation or prediction variance, and they can be classified according to global and local error metrics [31; 32; 33; 34]. [PERSON] et al. [33] constructed a WAS using several individual surrogate models for the optimization of alkaline-surfactant-polymer flooding processes, and found that the WAS exhibited better performance than individual surrogates. [PERSON] et al. [31] proposed different approaches in which the weights of the ensemble were determined based on the generalized mean square cross-validation error (GMSE). [PERSON] et al. [32] proposed an optimization method to calculate the weight factors by minimizing the GMSE. [PERSON] et al. [34] used a recursive process to obtain the weight factors, updating them in each iteration until the convergence goal was reached. The study described above shows that the weight factors are evaluated as a global metric. Unlike the global error metric, the local error metric method assesses weight factors in relatively small spaces, even point-by-point [35; 36; 37; 38; 39]. [PERSON] [35] used various local error measures to construct an ensemble model and presented a local error measure of pointwise cross-validation error. [PERSON] et al. [36] presented a v-nearest points cross-validation method to calculate the weight factors in a local region. Hierarchical design space reduction [37; 40], hybrid, and adaptive meta-modeling [38] are also effective methods based on local error metrics.