# An Ensemble of Adaptive Surrogate Models Based on Local Error Expectations

Authors: Huanwei Xu; Xin Zhang; Hao Li; Ge Xiang
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

An ensemble of surrogate models with high robustness and accuracy can effectively avoid the difficult choice of surrogate model. However, most of the existing ensembles of surrogate models are constructed with static sampling methods. In this paper, we propose an ensemble of adaptive surrogate models by applying adaptive sampling strategy based on expected local errors. In the proposed method, local error expectations of the surrogate models are calculated. Then according to local error expectations, the new sample points are added within the dominating radius of the samples. Constructed by the RBF and Kriging models, the ensemble of adaptive surrogate models is proposed by combining the adaptive sampling strategy. The benchmark test functions and an application problem that deals with driving arm base of palletizing robot show that the proposed method can effectively improve the global and local prediction accuracy of the surrogate model.


---

## Body

## 1. Introduction

In the engineering design problem, computer simulation is usually applied to replace the real physics experiments. For complex engineering problems, sometimes the performance function is implicit, or due to cost and time limit, the surrogate model is often applied to approximate the real physical model. Commonly used surrogate models mainly include Kriging [1], artificial neural network [2], radial basis function (RBF) [3], support vector regression(SVR) [4], and polynomial response surface(PRS) [5].When surrogate model is applied, how to find a suitable surrogate model is a difficult task. In order to improve the adaptability of the surrogate model, a reasonable choice is to use a linear weighted combination of different surrogate models, that is, an ensemble of surrogate models. Compared with the single surrogate model, an ensemble of surrogate models can save a lot of time wasted in screening the surrogate models. Many scholars have conducted in-depth research on it and have obtained many good achievements. Huang [6] found that the ensemble of surrogate models has higher prediction accuracy than the single surrogate model. Yan [7] proposed a new weight function construction method, which has the same accuracy as the optimal submodel and can improve the approximation of the true response distribution. Lu [8] found that the multisurrogate model has better optimization results than the single surrogate model’s. Pan [9] applied the ensemble of surrogate models to the lightweight design of the car body, and the results achieved a better optimization effect. Liu [10] established the ensemble of surrogate models to solve the structure optimization of car parts. Xing [11] assigned weights to three single surrogate models by using the adaptive metropolis-Markov chain Monte Carlo method. Yin [12] compared the application of a single surrogate model and an ensemble of surrogate models in groundwater restoration design optimization problems, and the results showed that the ensemble of surrogate models is more robust. Li [13] proposed a surrogate-assisted particle swarm algorithm, which can effectively balance the global search and local search. Donncha [14] successfully used the ensemble of surrogate models to improve the forecasting system with significant effects. Ouyang [15] used the analysis of variance method to determine the weights of ensemble of surrogate models. The comparison results show that the proposed method can not only improve the prediction performance of surrogate model, but also obtain a reliable solution. Chen [16] presented a new ensemble model which combines the advantages of global and local measures. The results show that the proposed ensemble model has satisfactory robustness and accuracy. Zhang [17] proposed a unified ensemble of surrogates with global and local measures for global metamodeling. It is concluded that the proposed model has superior accuracy while keeping comparable robustness and efficiency.Although some progress has been made in the research of the ensemble of surrogate models, most of the current methods for constructing the ensemble of surrogate models are stationary sampling. The problem with stationary sampling is that, in order to obtain an ensemble of surrogate models that meets the accuracy requirements, the sample size must be large enough. Adaptive sampling can obtain new samples that benefit the quality of the surrogate model, which can minimize the total sample size. However, the current adaptive sampling is often applied for a single surrogate model [18–21]. Only a few scholars combine the adaptive sampling strategy with the ensemble of surrogate models [22, 23]. The remainder of this paper is organized as follows. Section 2 briefly reviews the main steps to establish the ensemble of surrogate models. In Section 3, the ensemble of surrogate models using adaptive sampling strategy based on local error expectations is described. The proposed method is verified by numerical examples and compared with the three classical ensembles of surrogate models in Section 4. Section 5 applies the proposed method to the engineering design problem of driving arm base of palletizing robot. Finally, the conclusions are given.


## 2. Establishment of the Ensemble of Surrogate Models

There are three main steps to establish the ensemble of surrogate models:(1)
Design of experiment: the experiment design methods are applied to determine the spatial distribution of sample points. Experiment design methods mainly include Central Composite Designs (CCDs) [24], Orthogonal Design [25], and Latin Hypercube Design (LHD) [26]. LHD is the most popular sampling method due to good spatial uniformity. The experiment design method used in this paper is also LHD.(2)
Establishment of the ensemble of surrogate models: the surrogate models can be divided into two categories. One is interpolation methods, such as RBF and Kriging. For these methods, the prediction errors of the sample points are zeroes, which has good unbiasedness. The other is the noninterpolation methods, such as PRS and SVR. The noninterpolation methods have certain fitting capabilities, but the surrogate models do not go through all sample points. Therefore, enough sample points are needed to ensure the high accuracy of the surrogate models, which has extremely high uncertainty. In view of the advantages and disadvantages of different surrogate models, the most commonly used surrogate models are the RBF model and the Kriging model. In this paper, these two surrogate models are combined to establish the ensemble of surrogate models. The expression of the ensemble of surrogate models is as follows [27]:(1)y^ex=∑i=1Nωiy^ix,∑i=1Nωi=1.wherey⌢e is the predicted response value of the ensemble of surrogate models and N is the number of surrogate models. ωi is the ith weight coefficient. y^i is the predicted response value of the ith surrogate model. Generally speaking, the higher the prediction accuracy, the larger the weight coefficient of the corresponding surrogate model.(3)
Accuracy verification: accuracy verification of surrogate model mainly includes two aspects: global accuracy and local accuracy. root mean square error (RMSE) [28] and coefficient of determination (R2) [29] are two main global accuracy evaluation methods. The corresponding expressions are as follows:(2)RMSE=1n∑i=1nyi−y^i2,R2=1−∑i=1nyi−y^i2∑i=1nyi−y¯2,whereyi is the actual response value of the ith test sample and y^i is the predicted response value of the surrogate model of the ith test sample. y¯ is the mean value of the actual response value, and n is the size of test sample points. For RMSE, the smaller the value, the higher the global prediction accuracy. The range of R2 is not greater than 1. The value of R2 can be negative if the fitting quality of the surrogate model is extremely low. The closer the value of R2 to 1, the higher the accuracy of the global approximation of the surrogate model. Although RMSE can evaluate the prediction accuracy of the surrogate model, the magnitude of the specific problem greatly affects the value of RMSE, which is not as intuitive and easy to understand as R2. The global accuracy evaluation method applied in this paper is the coefficient of determination R2.The local prediction accuracy evaluation method is maximum absolute error (MAE). The expression of MAE is as follows:(3)MAE=maxyi−y^i.Similar to RMSE, the smaller the MAE, the higher the local prediction accuracy of the surrogate model. In this paper, MAE is also used to evaluate the local prediction accuracy of the surrogate model.


## 3. The Ensemble of Adaptive Surrogate Models Based on Local Error Expectations

The existing adaptive sampling strategy of sample points is mainly for a specific surrogate model, which has poor versatility. In addition, due to the inconsistency of the existing adaptive sampling strategies, it will be very complicated to combine the ensemble of surrogate models with the adaptive sampling strategy. In this section, a universal adaptive sampling strategy based on local errors is proposed. By combining the new adaptive sampling strategy, the method to construct the ensemble of surrogate models is proposed.


### 3.1. Adaptive Sampling Based on Local Error Expectations

Since Kriging and RBF models usually can provide good accuracy for fitting highly nonlinear behaviors, so these two surrogate models are used in general engineering problems. At present, the most commonly used adaptive sampling method is the maximin distance approach proposed by Johnson [30]. Jin and Chen [31] made corresponding improvements and proposed the Maximin Scaled Distance Approach. In this paper, we also propose a universal adaptive sampling strategy based on the local error expectations named LEE strategy for different surrogate models and it is proposed to serve the construction of the ensemble of adaptive surrogate models. The process is shown in Figure 1.Figure 1
The adaptive sampling process based on local error expectations.The following are main steps of the LEE strategy:(1)
Build an initial surrogate model. First, LHD is used to obtain the initial sample points and obtain their response values. Since high accuracy is not required at the beginning of sampling, for different dimensional surrogate models, the initial number of sample points can be5nd, 10nd, and 20nd (nd is the number of design variables).(2)
Calculate the expected valueE[AE] of the local error. Use the existing sample points and their response values to construct a surrogate model, and use cross-validation error method (LOO-leave one method) to obtain the local error of each point. The local error of ith sample point is evaluated by the absolute error AEi=y^i−yi. Then the local error expectation E AE] can be obtained by the following expression:(4)EAE=∑i=1nAEin.By using cross-validation error method, each sample point serves as a test point, and the other sample points serve as the sample points that constitute the surrogate model. When each sample point serves as the test point, it can reflect its importance for modeling and the uncertainty around the sample point’s location. The absolute error AEi can reflect the uncertainty around this location, and the expected absolute error E[AE] of all sample points can reflect the uncertainty of the overall sample points.(3)
Calculate the dominating radius of the sample points. Since the initial sample points determined by LHD have certain uniformity, the same radius can be set for each sample point.n sample points can divide the design space into n−1 part. In order to ensure that the radius of each sample point does not intersect as much as possible, we propose the concept of the dominating radius of the sample point. Rj is the dominating radius of the jth dimension coordinate of the sample point; the expression is as follows:(5)Rj=xjmax−xjminn−1,j=1,2,…,nd,wherend is the size of the dimension and xjmax and xjmin are the upper and lower bounds of the jth dimension. Then, R=R1,R2,…,Rnd is dominating radius of each sample point.(4)
Obtain new sample points. When AEi > E[AE], the prediction uncertainty near ith sample point is greater than the average prediction uncertainty of the existing sample points. It means the degree of nonlinearity near ith sample point is relatively large. So a sample point is randomly added within the dominating radius of ith sample point with equal probability. In order to avoid the added sample point being too close to the existing sample points, the sample point that meets the following condition is not added to the sample database:(6)X∗j−Xclosestj<Rj10,j=1,2,…,nd,whereX∗ stands for the point to be added and Xclosest represents the sample point closest to point X∗. Formula (6) means that if the sample points X∗ and Xclosest are too close, they will influence the condition of the correlation matrix of the surrogate model, so the added sample point should be invalid.(5)
If the value ofR2 is greater than the preset value η, the final surrogate model is obtained; otherwise update the surrogate model. The new acquired sample points are added to the sample database. The corresponding response values of these new sample points are calculated. Then the surrogate model is updated according to the current database of sample points. Calculate the determination coefficient R2. If the value of R2 is greater than the preset value η, the adaptive sampling process ends; otherwise, return to step 2.In order to illustrate the feasibility of LEE strategy, the one-dimensional test function in [32] is selected and its expression is(7)fx=6x−22sin12x−4,x∈0,1.Figures2–4 are initial Kriging model, the absolute errors, and the updated Kriging model. Figure 2 shows that the overall prediction accuracy of the initial Kriging surrogate model is low, and the local errors near point 5 and point 6 are very large. It can be seen from Figure 3 that errors of sample points 5 and 6 of the initial Kriging model exceed E[AE], so random sample points are added in the dominating radius of points 5 and 6. It can be seen from Figure 4 that the added Kriging surrogate model has higher prediction accuracy. After adding the sample points, the prediction error in this area is significantly reduced, and the prediction accuracy is higher, which proves the effectiveness and feasibility of adaptive sampling based on LEE strategy.Figure 2
The initial Kriging model based on 6 sample points.Figure 3
The absolute errors of samples of the initial Kriging model.Figure 4
The updated Kriging model with 2 new sample points.In order to prove the versatility of LEE strategy for different surrogate models, the RBF surrogate model is also constructed based on the existing sample points and their response values. Figures5–7 are initial RBF model, the absolute errors, and the updated RBF model. It can be seen from Figure 5 that the overall prediction accuracy of the initial RBF surrogate model is low, and the local errors near points 1 and 6 are the largest. It can be seen from Figure 6 that local errors of sample points 1 and 6 of the initial RBF model exceed E[AE], so random sample points are added in the dominating radius of sample points 1 and 6. It can be seen from Figure 7 that the overall prediction accuracy of updated RBF surrogate model with two new sample points has been greatly improved, which further proves the feasibility and versatility of adaptive sampling based on LEE strategy.Figure 5
The initial RBF model based on 6 sample points.Figure 6
The absolute errors of samples of the initial RBF surrogate model.Figure 7
The updated RBF surrogate model with 2 new sample points.The proposed LEE strategy is also compared with another adaptive sampling strategy called the Maximin Scaled Distance Approach (MSDA) [31] through the classic test functions. The specific information of the test functions is shown in Table 1.Table 1
Test function expression.
Test functionDimensionTest function expressionBranin (BN)2fx=x2−5.1/4π2x12+5/πx1−62+101−1/8πcosx1+10x1∈−5,10,x2∈0,15Hartmann3 (H3)3fx=−∑i=14αiexp−∑j=13Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=3.010300.110353.010300.11035P=10−436891170267346994387747010918732554738157438828xi∈0,1Colville (CV)4fx=100x12−x22+x1−12+x3−12+90x32−x42+10.1x2−12+x4−12+19.8x2−1x4−1xi∈−10,10,i=1,2Six-Hump Camel (SHC)2fx=4−2.1x12+x14/3x12+x1x2+−4+4x22x22x1∈−3,3,x2∈−2,2The initial Kriging and RBF surrogate models are established, respectively, according to a certain number of initial sample points. The proposed LEE strategy and MSDA are applied to improve the accuracy of surrogate models. The convergence condition isR2 > 0.8. Comparison results of Kriging and RBF surrogate models are listed in Table 2.Table 2
Comparison results of Kriging surrogate model.
Test functionApproachThe number of initial samplesKriging modelRBF modelThe number of total samplesFinal value ofR2The number of total samplesFinal value ofR2BNLEE10180.946150.908MSDA270.899270.873H3LEE15260.896350.902MSDA340.837390.879CVLEE20290.909300.934MSDA440.943410.901SHCLEE10250.920210.941MSDA360.883290.866It can be seen from Table2 that when the numbers of initial sample points of the two methods are the same, the numbers of total sample points used by LEE strategy are less than MSDA’s. At the same time, except for CN function, the final values of R2 of the LEE strategy are greater than those of the MSDA in most functions, which means that surrogate models constructed by LEE strategy can achieve higher prediction accuracy than those constructed by MSDA.


### 3.2. The Ensemble of Adaptive Surrogate Models

In this section we construct the ensemble of surrogate models with LEE strategy. The flowchart is shown in Figure8.Figure 8
The construction of the ensemble of adaptive surrogate model based on LEE strategy.The main steps are as follows:(1)
Build Kriging and RBF surrogate models. Existing researches [8–12] prove that, in most cases, interpolation type (Kriging and RBF) surrogate models are more suitable for engineering problems. Therefore, this paper chooses Kriging and RBF models to form the ensemble of surrogate models. Construct Kriging and RBF models by using the initial sample points. Then, obtain the predicted error sum of square (PRESS) [33], MAE, and R2 values of Kriging and RBF models by applying CV verification method (LOO-leave one method). The absolute errors (AEs) of each sample point of Kriging and RBF models are calculated. Since Forrester [34] has already proved that the surrogate model has better predictive ability when the coefficient of determination R2 is greater than 0.8, we use R2 > 0.8 as convergence conditions.(2)
Obtain new sample points. The new sample points are generated by applying adaptive sampling method based on LEE strategy. The sample database is updated.(3)
Update the Kriging and the RBF models. Calculate the true response values of the newly added sample points and reconstruct the Kriging and the RBF models. As long as theR2 of one of the two surrogate models is greater than 0.8, the operation of adding sample points is ended, and the final Kriging model and RBF model are obtained. Otherwise return to step 2.(4)
Calculate the weight coefficients of the Kriging and the RBF models and get the final ensemble of adaptive surrogate models. Cross validation (CV) [35] is performed to obtain the respective PRESS values of Kriging and RBF models. When there are n sample points in the database, all sample points except the ith point are used to construct the single surrogate model, and the ith point is used as a test point. The prediction error of the ith sample point is(8)ei=yi−y^−i,where yi is the true response value of the ith sample point and y^−i is the predicted response value of the ith sample point in the single surrogate model composed of all sample points except ith sample point. The prediction sum of squares is the sum of the prediction errors of all sample points, as shown in the following formula:(9)PRESS=∑i=1nei2.The weight coefficient corresponding to each single surrogate model is calculated by the inverse proportional averaging method, and the weight coefficient calculation formula is(10)ωi=1/Pi∑j=1N1/Pj,where Pi is the PRESS value at the ith sample point. In this paper, N is equal to 2. Then the final ensemble of adaptive surrogate models is obtained by linearly weighting each surrogate model.


## 3.1. Adaptive Sampling Based on Local Error Expectations

Since Kriging and RBF models usually can provide good accuracy for fitting highly nonlinear behaviors, so these two surrogate models are used in general engineering problems. At present, the most commonly used adaptive sampling method is the maximin distance approach proposed by Johnson [30]. Jin and Chen [31] made corresponding improvements and proposed the Maximin Scaled Distance Approach. In this paper, we also propose a universal adaptive sampling strategy based on the local error expectations named LEE strategy for different surrogate models and it is proposed to serve the construction of the ensemble of adaptive surrogate models. The process is shown in Figure 1.Figure 1
The adaptive sampling process based on local error expectations.The following are main steps of the LEE strategy:(1)
Build an initial surrogate model. First, LHD is used to obtain the initial sample points and obtain their response values. Since high accuracy is not required at the beginning of sampling, for different dimensional surrogate models, the initial number of sample points can be5nd, 10nd, and 20nd (nd is the number of design variables).(2)
Calculate the expected valueE[AE] of the local error. Use the existing sample points and their response values to construct a surrogate model, and use cross-validation error method (LOO-leave one method) to obtain the local error of each point. The local error of ith sample point is evaluated by the absolute error AEi=y^i−yi. Then the local error expectation E AE] can be obtained by the following expression:(4)EAE=∑i=1nAEin.By using cross-validation error method, each sample point serves as a test point, and the other sample points serve as the sample points that constitute the surrogate model. When each sample point serves as the test point, it can reflect its importance for modeling and the uncertainty around the sample point’s location. The absolute error AEi can reflect the uncertainty around this location, and the expected absolute error E[AE] of all sample points can reflect the uncertainty of the overall sample points.(3)
Calculate the dominating radius of the sample points. Since the initial sample points determined by LHD have certain uniformity, the same radius can be set for each sample point.n sample points can divide the design space into n−1 part. In order to ensure that the radius of each sample point does not intersect as much as possible, we propose the concept of the dominating radius of the sample point. Rj is the dominating radius of the jth dimension coordinate of the sample point; the expression is as follows:(5)Rj=xjmax−xjminn−1,j=1,2,…,nd,wherend is the size of the dimension and xjmax and xjmin are the upper and lower bounds of the jth dimension. Then, R=R1,R2,…,Rnd is dominating radius of each sample point.(4)
Obtain new sample points. When AEi > E[AE], the prediction uncertainty near ith sample point is greater than the average prediction uncertainty of the existing sample points. It means the degree of nonlinearity near ith sample point is relatively large. So a sample point is randomly added within the dominating radius of ith sample point with equal probability. In order to avoid the added sample point being too close to the existing sample points, the sample point that meets the following condition is not added to the sample database:(6)X∗j−Xclosestj<Rj10,j=1,2,…,nd,whereX∗ stands for the point to be added and Xclosest represents the sample point closest to point X∗. Formula (6) means that if the sample points X∗ and Xclosest are too close, they will influence the condition of the correlation matrix of the surrogate model, so the added sample point should be invalid.(5)
If the value ofR2 is greater than the preset value η, the final surrogate model is obtained; otherwise update the surrogate model. The new acquired sample points are added to the sample database. The corresponding response values of these new sample points are calculated. Then the surrogate model is updated according to the current database of sample points. Calculate the determination coefficient R2. If the value of R2 is greater than the preset value η, the adaptive sampling process ends; otherwise, return to step 2.In order to illustrate the feasibility of LEE strategy, the one-dimensional test function in [32] is selected and its expression is(7)fx=6x−22sin12x−4,x∈0,1.Figures2–4 are initial Kriging model, the absolute errors, and the updated Kriging model. Figure 2 shows that the overall prediction accuracy of the initial Kriging surrogate model is low, and the local errors near point 5 and point 6 are very large. It can be seen from Figure 3 that errors of sample points 5 and 6 of the initial Kriging model exceed E[AE], so random sample points are added in the dominating radius of points 5 and 6. It can be seen from Figure 4 that the added Kriging surrogate model has higher prediction accuracy. After adding the sample points, the prediction error in this area is significantly reduced, and the prediction accuracy is higher, which proves the effectiveness and feasibility of adaptive sampling based on LEE strategy.Figure 2
The initial Kriging model based on 6 sample points.Figure 3
The absolute errors of samples of the initial Kriging model.Figure 4
The updated Kriging model with 2 new sample points.In order to prove the versatility of LEE strategy for different surrogate models, the RBF surrogate model is also constructed based on the existing sample points and their response values. Figures5–7 are initial RBF model, the absolute errors, and the updated RBF model. It can be seen from Figure 5 that the overall prediction accuracy of the initial RBF surrogate model is low, and the local errors near points 1 and 6 are the largest. It can be seen from Figure 6 that local errors of sample points 1 and 6 of the initial RBF model exceed E[AE], so random sample points are added in the dominating radius of sample points 1 and 6. It can be seen from Figure 7 that the overall prediction accuracy of updated RBF surrogate model with two new sample points has been greatly improved, which further proves the feasibility and versatility of adaptive sampling based on LEE strategy.Figure 5
The initial RBF model based on 6 sample points.Figure 6
The absolute errors of samples of the initial RBF surrogate model.Figure 7
The updated RBF surrogate model with 2 new sample points.The proposed LEE strategy is also compared with another adaptive sampling strategy called the Maximin Scaled Distance Approach (MSDA) [31] through the classic test functions. The specific information of the test functions is shown in Table 1.Table 1
Test function expression.
Test functionDimensionTest function expressionBranin (BN)2fx=x2−5.1/4π2x12+5/πx1−62+101−1/8πcosx1+10x1∈−5,10,x2∈0,15Hartmann3 (H3)3fx=−∑i=14αiexp−∑j=13Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=3.010300.110353.010300.11035P=10−436891170267346994387747010918732554738157438828xi∈0,1Colville (CV)4fx=100x12−x22+x1−12+x3−12+90x32−x42+10.1x2−12+x4−12+19.8x2−1x4−1xi∈−10,10,i=1,2Six-Hump Camel (SHC)2fx=4−2.1x12+x14/3x12+x1x2+−4+4x22x22x1∈−3,3,x2∈−2,2The initial Kriging and RBF surrogate models are established, respectively, according to a certain number of initial sample points. The proposed LEE strategy and MSDA are applied to improve the accuracy of surrogate models. The convergence condition isR2 > 0.8. Comparison results of Kriging and RBF surrogate models are listed in Table 2.Table 2
Comparison results of Kriging surrogate model.
Test functionApproachThe number of initial samplesKriging modelRBF modelThe number of total samplesFinal value ofR2The number of total samplesFinal value ofR2BNLEE10180.946150.908MSDA270.899270.873H3LEE15260.896350.902MSDA340.837390.879CVLEE20290.909300.934MSDA440.943410.901SHCLEE10250.920210.941MSDA360.883290.866It can be seen from Table2 that when the numbers of initial sample points of the two methods are the same, the numbers of total sample points used by LEE strategy are less than MSDA’s. At the same time, except for CN function, the final values of R2 of the LEE strategy are greater than those of the MSDA in most functions, which means that surrogate models constructed by LEE strategy can achieve higher prediction accuracy than those constructed by MSDA.


## 3.2. The Ensemble of Adaptive Surrogate Models

In this section we construct the ensemble of surrogate models with LEE strategy. The flowchart is shown in Figure8.Figure 8
The construction of the ensemble of adaptive surrogate model based on LEE strategy.The main steps are as follows:(1)
Build Kriging and RBF surrogate models. Existing researches [8–12] prove that, in most cases, interpolation type (Kriging and RBF) surrogate models are more suitable for engineering problems. Therefore, this paper chooses Kriging and RBF models to form the ensemble of surrogate models. Construct Kriging and RBF models by using the initial sample points. Then, obtain the predicted error sum of square (PRESS) [33], MAE, and R2 values of Kriging and RBF models by applying CV verification method (LOO-leave one method). The absolute errors (AEs) of each sample point of Kriging and RBF models are calculated. Since Forrester [34] has already proved that the surrogate model has better predictive ability when the coefficient of determination R2 is greater than 0.8, we use R2 > 0.8 as convergence conditions.(2)
Obtain new sample points. The new sample points are generated by applying adaptive sampling method based on LEE strategy. The sample database is updated.(3)
Update the Kriging and the RBF models. Calculate the true response values of the newly added sample points and reconstruct the Kriging and the RBF models. As long as theR2 of one of the two surrogate models is greater than 0.8, the operation of adding sample points is ended, and the final Kriging model and RBF model are obtained. Otherwise return to step 2.(4)
Calculate the weight coefficients of the Kriging and the RBF models and get the final ensemble of adaptive surrogate models. Cross validation (CV) [35] is performed to obtain the respective PRESS values of Kriging and RBF models. When there are n sample points in the database, all sample points except the ith point are used to construct the single surrogate model, and the ith point is used as a test point. The prediction error of the ith sample point is(8)ei=yi−y^−i,where yi is the true response value of the ith sample point and y^−i is the predicted response value of the ith sample point in the single surrogate model composed of all sample points except ith sample point. The prediction sum of squares is the sum of the prediction errors of all sample points, as shown in the following formula:(9)PRESS=∑i=1nei2.The weight coefficient corresponding to each single surrogate model is calculated by the inverse proportional averaging method, and the weight coefficient calculation formula is(10)ωi=1/Pi∑j=1N1/Pj,where Pi is the PRESS value at the ith sample point. In this paper, N is equal to 2. Then the final ensemble of adaptive surrogate models is obtained by linearly weighting each surrogate model.


## 4. Numerical Example Analysis

In order to verify the versatility and effectiveness of the ensemble of adaptive surrogate models based on local error expectations, we compare the proposed method (ensemble of adaptive surrogate model, EOASM) with three typical ensemble of surrogate model construction methods: PRESS method, BestPRESS method, and PWS (PRESS Weighted Surrogate) method [36].Among the three most widely used methods for constructing an ensemble of surrogate model, the most classic one is to use PRESS as a measure of the weight coefficient calculation. If the PRESS value of a certain surrogate model is larger, the weight coefficient is smaller, also known as an inverse proportional averaging method, and its weight coefficient calculation formula is(11)ωi=1/Pi∑j=1N1/Pj.The BestPRESS method selects the single surrogate model with the smallest PRESS value as the final surrogate model, which is essentially a single surrogate model. Another method is the heuristic calculation weight coefficient algorithm proposed by Goel [36], and its calculation formula is(12)ωi=ωi∗∑j=1nωj∗,where ωi∗=Ei+αEavgβ and Eavg=∑j=1nEj/n. Ei is the PRESS of the ith surrogate model. The recommended parameter values are α=0.05,β=−1.


### 4.1. Benchmark Functions

In this paper, six benchmark functions from low dimension to high dimension are selected. The information of benchmark functions is shown in Table3.Table 3
Test function expression.
Test function expressionDimensionTest function expressionBranin2fx=x2−5.1/4π2x12+5/πx1−62+101−1/8πcosx1+10x1∈−5,10,x2∈0,15Hartmann-33fx=−∑i=14αiexp−∑j=13Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=3.010300.110353.010300.11035P=10−436891170267346994387747010918732554738157438828xi∈0,1Hartmann-44fx=1/0.8391.1−∑i=14αiexp−∑j=14Aijxj−Pij2|α=1.0,1.2,3.0,3.2TA=103173.51.780.0510170.181433.51.7101781780.05100.114P=10−41312169655691248283588623294135830737361004999123481451352228833047665040478828873257431091381xi∈0,1Hartmann-66fx=−∑i=14αiexp−∑j=16Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=103173.51.780.0510170.181433.51.7101781780.05100.114P=10−41312169655691248283588623294135830737361004999123481451352228833047665040478828873257431091381xi∈0,1Styblinski-Tang88fx=1/2∑i=18xi4−16xi2+5xi, xi∈−5,5Styblinski-Tang1010fx=1/2∑i=110xi4−16xi2+5xi, xi∈−5,5The Branin, Hartmann-3, and Hartmann-4 functions are low-dimensional. Latin hypercube sampling with 5n sample points is enough, which meet the accuracy requirements. Since the Hartmann-6, Styblinski-Tang8, and Styblinski-Tang10 are high dimensional, the Latin hypercube sampling with 20n sample points is used.


### 4.2. The Analysis of Global Prediction Accuracy

The global prediction accuracies of different ensembles of surrogate models are compared. The total number of samples is recorded when the EOASM method reaches the convergence condition. For the other three ensembles of surrogate models constructed by the PRESS method, BestPRESS method, and PWS method, the Latin hypercube sampling method is used to generate the same total sample size. So the number of sample points in the four methods is the same. After 20 comparative experiments, the average values of the determination of coefficientR2 of each ensemble of surrogate models are shown in Table 4.Table 4
Mean values ofR2.
Benchmark test functionTotal samplePRESSBestPRESSPWSEOASMBranin150.73510.70890.73640.9446Hartmann-3230.69340.66430.69350.9007Hartmann-4300.65490.58470.65470.9313Hartmann-61890.68840.66120.68830.9797Styblinski-Tang82400.43100.39030.44130.9514Styblinski-Tang102990.29310.25880.29310.9624It can be seen from Table4 that when the total number of sample points is the same, the prediction accuracy of the ensemble of surrogate model constructed by the EOASM method is the highest. For example, for the Branin function, the average value of determination coefficient R2 of EOASM is 0.9446. Among the other three ensembles of surrogate models, the PRESS method has the largest average value of R2, which is much lower than that of the EOASM method. The results of the other test functions are similar to the Branin function.


### 4.3. The Analysis of Local Prediction Accuracy

The maximum absolute error (MAE) is used to evaluate the local accuracy. The maximum absolute error of the ensemble of surrogate model constructed by each method is compared when the number of sample points is the same. Table5 shows the mean values of MAE of different ensembles of surrogate models.Table 5
Mean values of MAE.
Benchmark test functionTotal samplePRESSBestPRESSPWSEOASMBranin15108.827283.2561108.797538.8241Hartmann-3231.25061.17221.25160.9772Hartmann-4302.25122.06962.25340.7489Hartmann-61890.45210.60210.45240.1578Styblinski-Tang8240370.6343331.6834370.541191.3723Styblinski-Tang10299330.5281308.0860330.761576.8565It can be seen from 6 benchmark functions that EOASM method has the smallest average value of the MAE among four ensembles of surrogate models, which means that the proposed method has the highest predict accuracy among four methods.


### 4.4. Robustness Analysis

Robustness is an important indicator for evaluating surrogate models. The robustness refers to the insensitivity of the prediction accuracy of the surrogate model to random sampling of sample points. In order to compare the robustness of each surrogate model intuitively, 20 sampling experiments are performed for each benchmark function. The distribution results of the determination coefficientR2 are presented in box plot [37], which are shown in Figure 9.Figure 9
R2 box diagram of the ensembles of surrogate models. (a) Branin function, (b) Hartmann-3 function, (c) Hartmann-4 function, (d) Hartmann-6 function, (e) Styblinski-Tang8 function, and (f) Styblinski-Tang10 function.
(a)(b)(c)(d)(e)(f)In Figure9, the box length indicates whether the surrogate model's determination coefficient R2 fluctuates greatly. The smaller the box length, the stronger the robustness of the surrogate model. It can be clearly seen that the box length of the ensemble of surrogate model constructed by the EOASM method is the shortest in each benchmark function, which indicates the EOASM method has the strongest robustness.


## 4.1. Benchmark Functions

In this paper, six benchmark functions from low dimension to high dimension are selected. The information of benchmark functions is shown in Table3.Table 3
Test function expression.
Test function expressionDimensionTest function expressionBranin2fx=x2−5.1/4π2x12+5/πx1−62+101−1/8πcosx1+10x1∈−5,10,x2∈0,15Hartmann-33fx=−∑i=14αiexp−∑j=13Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=3.010300.110353.010300.11035P=10−436891170267346994387747010918732554738157438828xi∈0,1Hartmann-44fx=1/0.8391.1−∑i=14αiexp−∑j=14Aijxj−Pij2|α=1.0,1.2,3.0,3.2TA=103173.51.780.0510170.181433.51.7101781780.05100.114P=10−41312169655691248283588623294135830737361004999123481451352228833047665040478828873257431091381xi∈0,1Hartmann-66fx=−∑i=14αiexp−∑j=16Aijxj−Pij2α=1.0,1.2,3.0,3.2TA=103173.51.780.0510170.181433.51.7101781780.05100.114P=10−41312169655691248283588623294135830737361004999123481451352228833047665040478828873257431091381xi∈0,1Styblinski-Tang88fx=1/2∑i=18xi4−16xi2+5xi, xi∈−5,5Styblinski-Tang1010fx=1/2∑i=110xi4−16xi2+5xi, xi∈−5,5The Branin, Hartmann-3, and Hartmann-4 functions are low-dimensional. Latin hypercube sampling with 5n sample points is enough, which meet the accuracy requirements. Since the Hartmann-6, Styblinski-Tang8, and Styblinski-Tang10 are high dimensional, the Latin hypercube sampling with 20n sample points is used.


## 4.2. The Analysis of Global Prediction Accuracy

The global prediction accuracies of different ensembles of surrogate models are compared. The total number of samples is recorded when the EOASM method reaches the convergence condition. For the other three ensembles of surrogate models constructed by the PRESS method, BestPRESS method, and PWS method, the Latin hypercube sampling method is used to generate the same total sample size. So the number of sample points in the four methods is the same. After 20 comparative experiments, the average values of the determination of coefficientR2 of each ensemble of surrogate models are shown in Table 4.Table 4
Mean values ofR2.
Benchmark test functionTotal samplePRESSBestPRESSPWSEOASMBranin150.73510.70890.73640.9446Hartmann-3230.69340.66430.69350.9007Hartmann-4300.65490.58470.65470.9313Hartmann-61890.68840.66120.68830.9797Styblinski-Tang82400.43100.39030.44130.9514Styblinski-Tang102990.29310.25880.29310.9624It can be seen from Table4 that when the total number of sample points is the same, the prediction accuracy of the ensemble of surrogate model constructed by the EOASM method is the highest. For example, for the Branin function, the average value of determination coefficient R2 of EOASM is 0.9446. Among the other three ensembles of surrogate models, the PRESS method has the largest average value of R2, which is much lower than that of the EOASM method. The results of the other test functions are similar to the Branin function.


## 4.3. The Analysis of Local Prediction Accuracy

The maximum absolute error (MAE) is used to evaluate the local accuracy. The maximum absolute error of the ensemble of surrogate model constructed by each method is compared when the number of sample points is the same. Table5 shows the mean values of MAE of different ensembles of surrogate models.Table 5
Mean values of MAE.
Benchmark test functionTotal samplePRESSBestPRESSPWSEOASMBranin15108.827283.2561108.797538.8241Hartmann-3231.25061.17221.25160.9772Hartmann-4302.25122.06962.25340.7489Hartmann-61890.45210.60210.45240.1578Styblinski-Tang8240370.6343331.6834370.541191.3723Styblinski-Tang10299330.5281308.0860330.761576.8565It can be seen from 6 benchmark functions that EOASM method has the smallest average value of the MAE among four ensembles of surrogate models, which means that the proposed method has the highest predict accuracy among four methods.


## 4.4. Robustness Analysis

Robustness is an important indicator for evaluating surrogate models. The robustness refers to the insensitivity of the prediction accuracy of the surrogate model to random sampling of sample points. In order to compare the robustness of each surrogate model intuitively, 20 sampling experiments are performed for each benchmark function. The distribution results of the determination coefficientR2 are presented in box plot [37], which are shown in Figure 9.Figure 9
R2 box diagram of the ensembles of surrogate models. (a) Branin function, (b) Hartmann-3 function, (c) Hartmann-4 function, (d) Hartmann-6 function, (e) Styblinski-Tang8 function, and (f) Styblinski-Tang10 function.
(a)(b)(c)(d)(e)(f)In Figure9, the box length indicates whether the surrogate model's determination coefficient R2 fluctuates greatly. The smaller the box length, the stronger the robustness of the surrogate model. It can be clearly seen that the box length of the ensemble of surrogate model constructed by the EOASM method is the shortest in each benchmark function, which indicates the EOASM method has the strongest robustness.


## 5. Engineering Application

In the design of the palletizing robot, the design of the driving arm base plays a key role. The overall assembly of the palletizing robot is shown in Figure10.Figure 10
Overall assembly drawing of palletizing robot.The driving arm base bears large load. When it is assembled with the boom, it will deform to a certain extent, which will cause strain and stress. However, these physical quantities are difficult to express using explicit functions. It is often necessary to obtain their data through a large number of simulation tests. The specific material properties are shown in Table6.Table 6
Material properties of QT500-7.
Physical quantityUnitValueDensitykg/m37 × 103Elastic modulusPa1.62 × 1011Poisson’s ratio—0.28Yield strengthPa3.2 × 108Tensile strengthPa5 × 108Shear modulusPa6.27 × 1010The structure of the driving arm base is shown in Figure11. Considering the assembly relationship of each part, four nonassembly dimensions are selected as design variables, which are shown in Table 7. When the force and torque of the driving arm base reach the maximum, the generated stress is the largest. The fatigue damage is more likely to be caused. Power is carried out through UG software simulation to obtain the maximum force and torque of the assembly hole of the driving arm base. The curve of the force and torque with time is shown in Figure 12. It can be seen that, at 3 seconds, the driving arm base bears the maximum force and the maximum torque.Figure 11
Driving arm base of palletizing robot.Table 7
Design variables of driving arm base.
Design variablesNameUnitRangesx1Thickness of front platemm13–18x2Thickness of back platemm8–13x3Thickness of left and right boardmm20–25x4Thickness of ribmm8–13Figure 12
The force and torque of the driving arm base. (a) The force changes with time. (b) The torque changes with time.
(a)(b)Since the maximum stress is difficult to calculate directly, it is selected as the object function, and its true response value is obtained by simulation with Ansys finite element software, as shown in Figure13.Figure 13
Stress cloud diagram of driving arm base.The proposed method in this paper is used to construct the ensemble of surrogate model of maximum stress. The Latin hypercube sampling is initially adopted. The number of initial sample points is 10nd, which is 40 sample points.The values of global accuracy evaluation indexR2 and the local accuracy evaluation index MAE of surrogate model constructed by the EOASM method are shown in Table 8. It can be seen that the number of total sample points after convergence is 60. The CPU of the simulation platform is Intel Core i5-4590 3.30 GHz, the memory is 16G, and the operating system is Windows 10. It takes 6 minutes to perform a static structural simulation. The traditional design requires thousands of simulation experiments to roughly find the optimal value; optimization based on surrogate model only requires 60 simulation experiments, which greatly reduces computational cost of the simulation. The initial value of R2 increases from 0.3822 to 0.8979. The global prediction accuracy is increased by 135%. Meanwhile, the value of MAE reduces from 4.1565 to 0.5007. The local prediction accuracy is significantly improved. In summary, the EOASM method has good applicability to engineering problems and can greatly reduce the calculation cost of physical experiments.Table 8
Prediction accuracy of ensemble of surrogate model constructed by EOASM method.
Evaluation perspectiveThe initial samplesThe total samplesInitial dataEOASM dataR2 average40600.38220.8979MAE average4.15650.5007


## 6. Conclusion

(1)
The adaptive sampling based on LEE strategy can greatly improve the prediction accuracy of the surrogate model based on as few sample points as possible, and it also has strong applicability to different types of surrogate models.(2)
The EOASM method based on LEE strategy can greatly improve the global prediction accuracy, local prediction accuracy, and the robustness of the ensemble of surrogate models.(3)
Although the prediction accuracy and robustness of the ensemble of surrogate models constructed by the EOASM method have been improved to some extent, it still has not escaped the high-dimensional curse of the surrogate model. Under the condition that the sample size is already large, it is possible that the accuracy of the surrogate model is extremely low. Therefore, the high-dimensional problem of the surrogate model is still a problem to be solved.



---

*Source: 8857417-2021-02-10.xml*