# Pile-Raft Settlements Prediction under Coupled Static-Dynamic Loads Using Four Heuristic Regression Approaches

Authors: Mosbeh R. Kaloop; Jong Wan Hu; Emad Elbeltagi
Journal: Shock and Vibration
 (2018)

---

## Abstract

One of the main driving factors for structures’ evaluation is the foundation settlement. Measuring structures’ settlement in field is costly especially when heavy loads are applied. Settlement prediction models can be used to avoid the high cost of settlement field tests. Four advanced heuristic regression methods are developed and applied in this study to estimate raft foundations’ settlement, namely, multivariate adaptive regression splines (MARS), M5 model tree (M5Tree), generalized regression neural networks (GRNN), and support vector regression (SVR) techniques. Simulation of raft pile foundations is utilized to calculate the settlements of piles under the effect of static and dynamic loads. Previous studies are compared with the newly developed models. The results show that the four models can be used to accurately predict foundations’ settlements in the training stage. Also, the results reveal that the MARS and SVR models performed slightly better than the M5Tree and GRNN models in the testing stage and accordingly can be used to predict foundations’ settlement. The SVR model outperformed other models when few numbers of measurements are available.


---

## Body

## 1. Introduction

Piles foundations are usually used to decrease the settlements of heavy loads structures. Monitoring or predicting piles foundations settlement of important structures (e.g., long-span bridges, and skyscrapers) is an essential task to ensure their safety. Understanding the behavior of foundations buried in soil is complex and not defined completely yet [1]. The soil characteristics, pile geometry, and the loads are the main factors for the settlement of foundations [2–5]. Principles of pile foundation and settlement analysis are described in [6]. The geometry of raft pile foundations is a common factor for the settlement of piles [5]. The geometry of piles foundation is defined by the length, diameter, number, and distribution of piles [5]. In this paper, prediction of raft pile foundations’ settlement is investigated under both static and dynamic loads.In the literature, many studies utilized different methods to study the behavior and predict the settlement of foundations subjected to different types of loads and with different foundations shapes [1–3, 5, 7]. Detection of piles capacity and settlement using field tests is costly and takes long time. Therefore, computation techniques can be used to detect the settlement of piles [1]. In this study, a new model is developed to predict the settlement of raft foundations using four different heuristic regression methods.In the literature, many numerical models are applied for the foundations settlement prediction under static and static-dynamic loads. Shahin [1] summarized previous studies that used the artificial intelligence techniques. Neural networks are utilized to estimate the piles’ settlement based on standard penetration tests data and the shaft geometry and Shahin [1] concludes that the correlation between measured and predicted values is 0.972. Dantas et al. [3] utilized back-propagation neural networks to estimate the settlement of piles under static loads; the correlation between measured and predicted settlements is 0.94. Ghorbani and Niavol [5] applied the neural networks and evolutionary polynomial regression models to estimate pile foundations’ settlement under static and static-dynamic loads. They concluded that the two methods can be used to accurately estimate settlements of piles with correlation 0.97 for the two models with the two loading cases. Alkroosh and Nikraz [8] utilized the genetic expression algorithm to identify the piles capacity based on dynamic loads, and they conclude that the genetic model is more accurate than the neural networks and the correlation between predicted and measured values is 0.96 for the training sets.On the other hand, regression computational methods are widely used to simplify the nonlinear prediction behavior for the engineering characteristics in many different areas [9–12]. Moreover, some applications of regression methods for the geotechnical engineering are presented in [13–15]. Instead their applications to predict settlement of pile-raft foundations under static-dynamic loads are still limited [16, 17]. In this study, the multivariate adaptive regression (MARS), M5 model tree (M5Tree), generalized regression neural networks (GRNN), and support vector regression (SVR) techniques are applied to predict the pile-raft foundations settlement under static-dynamic loads. Simulation models are used to evaluate pile-raft foundations behavior with different piles dimensions [18]. Moayed et al. [18] found that piles with equal diameters have higher settlement variation than those with unequal diameters. Moreover, Baziar et al. [19] evaluated the simulation model for a pile-raft foundation and compared the performance of this model with 1g physical model, and they found that the simulation model can be utilized to simulate the accurate behavior of settlement for pile-raft foundations. In this study, the results of the settlements extracted by a simulation model presented in [5] for pile-raft foundations are utilized to design a model for foundations’ settlement prediction.This study aims, mainly, to investigate the ability of regression models to predict pile-raft foundations’ settlement under coupled static and dynamic loads. Four regression models, MARS, M5Tree, GRNN, and SVR, are developed, assessed, and evaluated using statistical analysis. In addition, the obtained results are discussed and compared with the results of a previous study by Ghorbani and Niavol [5].


## 2. Data and Methods

### 2.1. Data Used

Figure1 illustrates the simulation model design with dynamic loads utilized; Figure 2 presents the data used and extracted from this simulation model. The results of this simulation model and its ability to accurately estimate the settlement values (S) are verified against the physical model results [19]. The simulation model designed by Ghorbani and Niavol [5] comprises a square raft foundation with different number of piles “N” (9, 16 and 25), piles’ diameter “d” (0.3 and 0.5 m), piles spacing “s”, and length of piles “l”. The static loads “P” are assumed to be 60 and 90 Kpa, as presented in Figure 2.Figure 1
(a) 9-pile foundation diagram and (b) affected dynamic load [5].
(a) (b)Figure 2
Input and output data used.Table1 shows the statistical analysis (maximum, minimum, and standard deviation (sd)) for the input parameters and settlement calculations. As per the data presented in Table 1, the s/d and l/d variations are 2 and 16, respectively, with high settlement variation value, 9.48 cm. Also, as the number of piles and their diameters change, small values for the settlements are reported [5]. The relative parameters are studies and evaluated using the autocorrelation function between the inputs (N, d, s/d, l/d, and p) and the output (S) parameters [5]. The correlation between N, d, s/d, l/d, and p and S are 91.1, 92.5, 92.2, 86.2, and 96.1 %, respectively. However, the most important parameter is the axial load, while the parameter l/d effectiveness is shown to be lower. In addition, the variation between the effectiveness of five parameters is shown small for the N, d, s/d, and l/d. Therefore, the settlements of pile-raft foundations can be considered affected by the five inputs.Table 1
Statistical analysis for the input and output parameters of the simulation model.
Variables N d (m) s/d l/d P (kPa) S (cm) Max 25.00 0.50 6.00 32.00 90.00 12.31  Min 9.00 0.30 4.00 16.00 60.00 2.83  sd 6.57 0.10 0.82 5.94 15.05 2.21


### 2.2. Settlements Predicting Models

In this study, predicting settlement for pile-raft foundations’ models under static and dynamic loads using heuristic methods is developed. Five input parameters (N, d, s/d, l/d, and p) are used. A brief description of each model is presented in the following subsections.


#### 2.2.1. Multivariate Adaptive Regression Spline

The Multivariate adaptive regression spline (MARS) is a nonlinear nonparametric regression approach introduced by Friedman [21] and Friedman and Roosen [22]. The MARS is utilized to predict piles drivability [23], but its application to predict piles settlement is still limited. the MARS constructs a simulation nonlinear relationship between the dependent and independent regression variables using piecewise linear splines [10]. The basis functions (BFs) are used to design the MARS models [23]. The BFs are defined in pair forms based on a knot to define an inflection region. This method involves forward and backward phases. In the forward phase, candidate knots are placed at random positions, within the range of each predictor variable, to define pairs of BFs. The knots and their corresponding pair of BFs are adapted, in each step, to minimize the sum-of-squares of the residual errors. This process of adding BFs continues until the maximum number is reached. In the backward phase, redundant BFs that made the least contributions are deleted. An open MARS source code [24] is adopted and used to perform the analysis presented in this paper.The MARS model for estimating pile-raft foundations’ settlement can be presented as follows:(1)yx=b0+∑i=0mbiBFiwhere,y(x) is the predicted settlement, b is unknown coefficients which can be estimated by the least square method, and m is the BF numbers in the final model that is estimated in a forward and backward stepwise process. BF is the basis function which is defined based on knot (t) from piecewise linear basis functions which follow the form max⁡(0,x-t) [23]. The model performance is improved using the backward process with the less computationally expensive method of Generalized Cross-Validation (GCV):(2)GCV=1/N∑i=1Nei21-m+d∗m-1/2/N2where d is a penalty for each BF included in the developed submodel, N is the number of training data, and e is the model error. The numerator represents the mean square error of the model in the training phase, penalized by the denominator which accounts for the variance increase if model complexity increases. The GCV penalizes both the number of BFs and the number of knots. More details for this method can be found in [10, 23].


#### 2.2.2. M5 Model Tree

Quinlan [25] introduced the M5 model tree (M5Tree) and it was then improved by Wang and Witten [26], who found that it is more accurate than regression trees to model a nonlinear approaches [25]. Many engineering applications used the M5Tree method to design prediction models [11, 27, 28].This study introduces the modeling for the settlement using M5Tree. In general, the M5Tree depends on a regression function at terminal nodes. The model constructs a regression tree recursively by the instance space; the splitting criterion is used to minimize the intrasubset variability in the values down from the root through the branch to the nodes. Moreover, the splitting was utilized to design the decision tree (Figure 3). Variability is measured using the standard deviation of the values that reach that node from the root through the branch with calculating the expected reduction in error as a result of testing each attribute at that node. The attribute minimizing the expected error is chosen. The splitting stops if the values of all instances that reach a node vary slightly or only a few instances remain. The standard deviation reduction (SDR) is calculated as follows:(3)SDR=sdT-∑TiT∗sdTiFigure 3
M5Tree, splitting the input space into subspaces and the resultant diagram [20].where T is the set of examples that reach the node,Ti is the sets that resulted from splitting the chosen attribute, and sd is the standard deviation. Details of the method and model design can be found in [10, 20].


#### 2.2.3. Generalized Regression Neural Networks

Nonlinear prediction models using family of radial basis function neural networks (RBFNN) are used successfully in many engineering applications [9, 29]. Generalized regression neural network (GRNN) is a paradigm of RBFNN and shares a special property; while that does not need iterative training, the weight vector between the RBF and the output can be fixed as the target vector [29]. Kisi [9] stated that the difference between GRNN and RBF is in the weight parameter generation. Therefore, the processing time of the GRNN model is shorter than that of the RBF model when large data are utilized. In this study, the GRNN model is used to estimate the settlement of pile-raft foundations; this method is proposed in [29]. The neural network model consists of three layers, input, hidden, and output. Neurons in the hidden layer are used to hold the input vector, and the weight between newly hidden and output layers is assigned the target value [9, 29, 30]. The prediction output can be expressed as follows:(4)yx=∑i=1Nhiwix∑i=1Nhiwhere h is the output of the hidden neurons (hi=exp⁡⁡[-Di2/2σ2]; Di2=x-uiTx-ui, where D is the distance between input (x) and training (u) vectors and σ is a constant controlling the size of the perceptive region) and w is the weight corresponding to input vector (x) and hidden output (i). The misclassified pattern with minimum activation at the output neuron is applied based on Hoya and Chambers [29] theory to remove the prediction outliers results.


#### 2.2.4. Support Vector Regression

The support vector regression (SVR) is introduced by Gunn [31] and proposed by Suykens and Vandewalle [32] as tool for nonlinear modeling of different behaviors of engineering application [11, 12, 33]. The SVR performance with small samples is more popular than traditional statistical methods to model the nonlinear of dynamic behavior [12]. This study investigates the use of VSR models for the settlement prediction of pile-raft foundations. The SVR model maps input data into a higher dimensional feature space via a nonlinear mapping, and then a linear regression problem is obtained and solved in this space [33]. The SVR is described using ε-insensitive loss function as follows [31]:(5)Lεy=0foryx-y<εyx-y-εotherwisewherey(x) and y are predicted and observed vectors and Lεy indicates that it does not penalize errors below ε. The nonlinear SVR solution is given by(6)maxα,α∗⁡Wα,α∗=maxα,α∗⁡∑i=1Nαi∗yi-ε-αiyi+ε-0.5∑i=1N∑j=1Nαi∗-αiαj∗-αjKxi,xjwith constraints(7)0≤α,α≤C,i=1,2,…N∑i=1Nαi∗-αiwhere K is a kernel function and C is a regularized constant determining the tradeoff between the training error and model fitness. Solving (6) with the constraints presented in (7) to calculate the Lagrange multiplier (α) and the regression function, we get the following:(8)yx=∑i=1Nαi-αi∗Kxi,x+bwhere b is a bias term. Based on the nature of quadratic programming, only a number of coefficients among αiandαi∗ will be nonzero, and the data points associated with them refer to support vector [33]. Many kernel functions can be used in this case; the Gaussian kernel function is the most popular function utilized to predict the nonlinear of dynamic behavior [33]; in this study the linear, polynomial, and Gaussian functions are evaluated.


#### 2.2.5. Model Design and Performance Evaluation

Based on Ghorbani and Niavol [5] results and the developed model, the five parameters (N, d, s/d, l/d, and p) are used for the settlement prediction of pile-raft foundations. One hundred and forty-four data samples are utilized in this study. The data set is divided into training 75% (107 samples) and testing 25% (37 samples) subsets. Table 2 shows the statistical analysis for the training and testing subsets. It is clear that the input data sets have the same statistical parameters evaluation, approximately, while the settlement parameters for the training and testing have some changes and this may be caused by some extrapolation in the predicting values.Table 2
Statistical analysis of the training and testing data samples.
Training N d (m) s/d l/d P (kPa) S (cm) Mean 16.48 0.40 5.00 23.03 74.86 6.65  Max 25.00 0.50 6.00 32.00 90.00 12.31  Min 9.00 0.30 4.00 16.00 60.00 2.83  sd 5.98 0.10 0.82 5.96 15.07 2.31  Testing N d (m) s/d l/d P (kPa) S (cm)  Mean 17.22 0.40 5.00 22.92 75.41 6.72  Max 25.00 0.50 6.00 32.00 90.00 11.35  Min 9.00 0.30 4.00 16.00 60.00 3.62  sd 8.11 0.10 0.82 5.94 15.20 1.91Figure4 shows the four stages of the prediction process for the four models. First, the input and output data are prepared and processed by removing the outlier’s data; second, the four models parameters are developed using training data set and evaluated using testing data set. In the third stage, the four models results are compared to select the best model that can be used to predict the settlement; then the selected model results are compared with the Ghorbani and Niavol [5] results.Figure 4
Models processing.To evaluate the models, three statistical measures are utilized: coefficient of determination (R2), route mean square error (RMSE), and mean absolute error (MAE):(9)R2=1-∑i=1Nyi-fi2∑i=1Nyi-y-2(10)RMSE=0.5∑i=1Nyi-fi2(11)MAE=1N∑i=1Nyi-fiwherey and f are the actual and predicted settlements values, y- is the mean value for the actual values, and N is the number of observations.


## 2.1. Data Used

Figure1 illustrates the simulation model design with dynamic loads utilized; Figure 2 presents the data used and extracted from this simulation model. The results of this simulation model and its ability to accurately estimate the settlement values (S) are verified against the physical model results [19]. The simulation model designed by Ghorbani and Niavol [5] comprises a square raft foundation with different number of piles “N” (9, 16 and 25), piles’ diameter “d” (0.3 and 0.5 m), piles spacing “s”, and length of piles “l”. The static loads “P” are assumed to be 60 and 90 Kpa, as presented in Figure 2.Figure 1
(a) 9-pile foundation diagram and (b) affected dynamic load [5].
(a) (b)Figure 2
Input and output data used.Table1 shows the statistical analysis (maximum, minimum, and standard deviation (sd)) for the input parameters and settlement calculations. As per the data presented in Table 1, the s/d and l/d variations are 2 and 16, respectively, with high settlement variation value, 9.48 cm. Also, as the number of piles and their diameters change, small values for the settlements are reported [5]. The relative parameters are studies and evaluated using the autocorrelation function between the inputs (N, d, s/d, l/d, and p) and the output (S) parameters [5]. The correlation between N, d, s/d, l/d, and p and S are 91.1, 92.5, 92.2, 86.2, and 96.1 %, respectively. However, the most important parameter is the axial load, while the parameter l/d effectiveness is shown to be lower. In addition, the variation between the effectiveness of five parameters is shown small for the N, d, s/d, and l/d. Therefore, the settlements of pile-raft foundations can be considered affected by the five inputs.Table 1
Statistical analysis for the input and output parameters of the simulation model.
Variables N d (m) s/d l/d P (kPa) S (cm) Max 25.00 0.50 6.00 32.00 90.00 12.31  Min 9.00 0.30 4.00 16.00 60.00 2.83  sd 6.57 0.10 0.82 5.94 15.05 2.21


## 2.2. Settlements Predicting Models

In this study, predicting settlement for pile-raft foundations’ models under static and dynamic loads using heuristic methods is developed. Five input parameters (N, d, s/d, l/d, and p) are used. A brief description of each model is presented in the following subsections.


### 2.2.1. Multivariate Adaptive Regression Spline

The Multivariate adaptive regression spline (MARS) is a nonlinear nonparametric regression approach introduced by Friedman [21] and Friedman and Roosen [22]. The MARS is utilized to predict piles drivability [23], but its application to predict piles settlement is still limited. the MARS constructs a simulation nonlinear relationship between the dependent and independent regression variables using piecewise linear splines [10]. The basis functions (BFs) are used to design the MARS models [23]. The BFs are defined in pair forms based on a knot to define an inflection region. This method involves forward and backward phases. In the forward phase, candidate knots are placed at random positions, within the range of each predictor variable, to define pairs of BFs. The knots and their corresponding pair of BFs are adapted, in each step, to minimize the sum-of-squares of the residual errors. This process of adding BFs continues until the maximum number is reached. In the backward phase, redundant BFs that made the least contributions are deleted. An open MARS source code [24] is adopted and used to perform the analysis presented in this paper.The MARS model for estimating pile-raft foundations’ settlement can be presented as follows:(1)yx=b0+∑i=0mbiBFiwhere,y(x) is the predicted settlement, b is unknown coefficients which can be estimated by the least square method, and m is the BF numbers in the final model that is estimated in a forward and backward stepwise process. BF is the basis function which is defined based on knot (t) from piecewise linear basis functions which follow the form max⁡(0,x-t) [23]. The model performance is improved using the backward process with the less computationally expensive method of Generalized Cross-Validation (GCV):(2)GCV=1/N∑i=1Nei21-m+d∗m-1/2/N2where d is a penalty for each BF included in the developed submodel, N is the number of training data, and e is the model error. The numerator represents the mean square error of the model in the training phase, penalized by the denominator which accounts for the variance increase if model complexity increases. The GCV penalizes both the number of BFs and the number of knots. More details for this method can be found in [10, 23].


### 2.2.2. M5 Model Tree

Quinlan [25] introduced the M5 model tree (M5Tree) and it was then improved by Wang and Witten [26], who found that it is more accurate than regression trees to model a nonlinear approaches [25]. Many engineering applications used the M5Tree method to design prediction models [11, 27, 28].This study introduces the modeling for the settlement using M5Tree. In general, the M5Tree depends on a regression function at terminal nodes. The model constructs a regression tree recursively by the instance space; the splitting criterion is used to minimize the intrasubset variability in the values down from the root through the branch to the nodes. Moreover, the splitting was utilized to design the decision tree (Figure 3). Variability is measured using the standard deviation of the values that reach that node from the root through the branch with calculating the expected reduction in error as a result of testing each attribute at that node. The attribute minimizing the expected error is chosen. The splitting stops if the values of all instances that reach a node vary slightly or only a few instances remain. The standard deviation reduction (SDR) is calculated as follows:(3)SDR=sdT-∑TiT∗sdTiFigure 3
M5Tree, splitting the input space into subspaces and the resultant diagram [20].where T is the set of examples that reach the node,Ti is the sets that resulted from splitting the chosen attribute, and sd is the standard deviation. Details of the method and model design can be found in [10, 20].


### 2.2.3. Generalized Regression Neural Networks

Nonlinear prediction models using family of radial basis function neural networks (RBFNN) are used successfully in many engineering applications [9, 29]. Generalized regression neural network (GRNN) is a paradigm of RBFNN and shares a special property; while that does not need iterative training, the weight vector between the RBF and the output can be fixed as the target vector [29]. Kisi [9] stated that the difference between GRNN and RBF is in the weight parameter generation. Therefore, the processing time of the GRNN model is shorter than that of the RBF model when large data are utilized. In this study, the GRNN model is used to estimate the settlement of pile-raft foundations; this method is proposed in [29]. The neural network model consists of three layers, input, hidden, and output. Neurons in the hidden layer are used to hold the input vector, and the weight between newly hidden and output layers is assigned the target value [9, 29, 30]. The prediction output can be expressed as follows:(4)yx=∑i=1Nhiwix∑i=1Nhiwhere h is the output of the hidden neurons (hi=exp⁡⁡[-Di2/2σ2]; Di2=x-uiTx-ui, where D is the distance between input (x) and training (u) vectors and σ is a constant controlling the size of the perceptive region) and w is the weight corresponding to input vector (x) and hidden output (i). The misclassified pattern with minimum activation at the output neuron is applied based on Hoya and Chambers [29] theory to remove the prediction outliers results.


### 2.2.4. Support Vector Regression

The support vector regression (SVR) is introduced by Gunn [31] and proposed by Suykens and Vandewalle [32] as tool for nonlinear modeling of different behaviors of engineering application [11, 12, 33]. The SVR performance with small samples is more popular than traditional statistical methods to model the nonlinear of dynamic behavior [12]. This study investigates the use of VSR models for the settlement prediction of pile-raft foundations. The SVR model maps input data into a higher dimensional feature space via a nonlinear mapping, and then a linear regression problem is obtained and solved in this space [33]. The SVR is described using ε-insensitive loss function as follows [31]:(5)Lεy=0foryx-y<εyx-y-εotherwisewherey(x) and y are predicted and observed vectors and Lεy indicates that it does not penalize errors below ε. The nonlinear SVR solution is given by(6)maxα,α∗⁡Wα,α∗=maxα,α∗⁡∑i=1Nαi∗yi-ε-αiyi+ε-0.5∑i=1N∑j=1Nαi∗-αiαj∗-αjKxi,xjwith constraints(7)0≤α,α≤C,i=1,2,…N∑i=1Nαi∗-αiwhere K is a kernel function and C is a regularized constant determining the tradeoff between the training error and model fitness. Solving (6) with the constraints presented in (7) to calculate the Lagrange multiplier (α) and the regression function, we get the following:(8)yx=∑i=1Nαi-αi∗Kxi,x+bwhere b is a bias term. Based on the nature of quadratic programming, only a number of coefficients among αiandαi∗ will be nonzero, and the data points associated with them refer to support vector [33]. Many kernel functions can be used in this case; the Gaussian kernel function is the most popular function utilized to predict the nonlinear of dynamic behavior [33]; in this study the linear, polynomial, and Gaussian functions are evaluated.


### 2.2.5. Model Design and Performance Evaluation

Based on Ghorbani and Niavol [5] results and the developed model, the five parameters (N, d, s/d, l/d, and p) are used for the settlement prediction of pile-raft foundations. One hundred and forty-four data samples are utilized in this study. The data set is divided into training 75% (107 samples) and testing 25% (37 samples) subsets. Table 2 shows the statistical analysis for the training and testing subsets. It is clear that the input data sets have the same statistical parameters evaluation, approximately, while the settlement parameters for the training and testing have some changes and this may be caused by some extrapolation in the predicting values.Table 2
Statistical analysis of the training and testing data samples.
Training N d (m) s/d l/d P (kPa) S (cm) Mean 16.48 0.40 5.00 23.03 74.86 6.65  Max 25.00 0.50 6.00 32.00 90.00 12.31  Min 9.00 0.30 4.00 16.00 60.00 2.83  sd 5.98 0.10 0.82 5.96 15.07 2.31  Testing N d (m) s/d l/d P (kPa) S (cm)  Mean 17.22 0.40 5.00 22.92 75.41 6.72  Max 25.00 0.50 6.00 32.00 90.00 11.35  Min 9.00 0.30 4.00 16.00 60.00 3.62  sd 8.11 0.10 0.82 5.94 15.20 1.91Figure4 shows the four stages of the prediction process for the four models. First, the input and output data are prepared and processed by removing the outlier’s data; second, the four models parameters are developed using training data set and evaluated using testing data set. In the third stage, the four models results are compared to select the best model that can be used to predict the settlement; then the selected model results are compared with the Ghorbani and Niavol [5] results.Figure 4
Models processing.To evaluate the models, three statistical measures are utilized: coefficient of determination (R2), route mean square error (RMSE), and mean absolute error (MAE):(9)R2=1-∑i=1Nyi-fi2∑i=1Nyi-y-2(10)RMSE=0.5∑i=1Nyi-fi2(11)MAE=1N∑i=1Nyi-fiwherey and f are the actual and predicted settlements values, y- is the mean value for the actual values, and N is the number of observations.


## 2.2.1. Multivariate Adaptive Regression Spline

The Multivariate adaptive regression spline (MARS) is a nonlinear nonparametric regression approach introduced by Friedman [21] and Friedman and Roosen [22]. The MARS is utilized to predict piles drivability [23], but its application to predict piles settlement is still limited. the MARS constructs a simulation nonlinear relationship between the dependent and independent regression variables using piecewise linear splines [10]. The basis functions (BFs) are used to design the MARS models [23]. The BFs are defined in pair forms based on a knot to define an inflection region. This method involves forward and backward phases. In the forward phase, candidate knots are placed at random positions, within the range of each predictor variable, to define pairs of BFs. The knots and their corresponding pair of BFs are adapted, in each step, to minimize the sum-of-squares of the residual errors. This process of adding BFs continues until the maximum number is reached. In the backward phase, redundant BFs that made the least contributions are deleted. An open MARS source code [24] is adopted and used to perform the analysis presented in this paper.The MARS model for estimating pile-raft foundations’ settlement can be presented as follows:(1)yx=b0+∑i=0mbiBFiwhere,y(x) is the predicted settlement, b is unknown coefficients which can be estimated by the least square method, and m is the BF numbers in the final model that is estimated in a forward and backward stepwise process. BF is the basis function which is defined based on knot (t) from piecewise linear basis functions which follow the form max⁡(0,x-t) [23]. The model performance is improved using the backward process with the less computationally expensive method of Generalized Cross-Validation (GCV):(2)GCV=1/N∑i=1Nei21-m+d∗m-1/2/N2where d is a penalty for each BF included in the developed submodel, N is the number of training data, and e is the model error. The numerator represents the mean square error of the model in the training phase, penalized by the denominator which accounts for the variance increase if model complexity increases. The GCV penalizes both the number of BFs and the number of knots. More details for this method can be found in [10, 23].


## 2.2.2. M5 Model Tree

Quinlan [25] introduced the M5 model tree (M5Tree) and it was then improved by Wang and Witten [26], who found that it is more accurate than regression trees to model a nonlinear approaches [25]. Many engineering applications used the M5Tree method to design prediction models [11, 27, 28].This study introduces the modeling for the settlement using M5Tree. In general, the M5Tree depends on a regression function at terminal nodes. The model constructs a regression tree recursively by the instance space; the splitting criterion is used to minimize the intrasubset variability in the values down from the root through the branch to the nodes. Moreover, the splitting was utilized to design the decision tree (Figure 3). Variability is measured using the standard deviation of the values that reach that node from the root through the branch with calculating the expected reduction in error as a result of testing each attribute at that node. The attribute minimizing the expected error is chosen. The splitting stops if the values of all instances that reach a node vary slightly or only a few instances remain. The standard deviation reduction (SDR) is calculated as follows:(3)SDR=sdT-∑TiT∗sdTiFigure 3
M5Tree, splitting the input space into subspaces and the resultant diagram [20].where T is the set of examples that reach the node,Ti is the sets that resulted from splitting the chosen attribute, and sd is the standard deviation. Details of the method and model design can be found in [10, 20].


## 2.2.3. Generalized Regression Neural Networks

Nonlinear prediction models using family of radial basis function neural networks (RBFNN) are used successfully in many engineering applications [9, 29]. Generalized regression neural network (GRNN) is a paradigm of RBFNN and shares a special property; while that does not need iterative training, the weight vector between the RBF and the output can be fixed as the target vector [29]. Kisi [9] stated that the difference between GRNN and RBF is in the weight parameter generation. Therefore, the processing time of the GRNN model is shorter than that of the RBF model when large data are utilized. In this study, the GRNN model is used to estimate the settlement of pile-raft foundations; this method is proposed in [29]. The neural network model consists of three layers, input, hidden, and output. Neurons in the hidden layer are used to hold the input vector, and the weight between newly hidden and output layers is assigned the target value [9, 29, 30]. The prediction output can be expressed as follows:(4)yx=∑i=1Nhiwix∑i=1Nhiwhere h is the output of the hidden neurons (hi=exp⁡⁡[-Di2/2σ2]; Di2=x-uiTx-ui, where D is the distance between input (x) and training (u) vectors and σ is a constant controlling the size of the perceptive region) and w is the weight corresponding to input vector (x) and hidden output (i). The misclassified pattern with minimum activation at the output neuron is applied based on Hoya and Chambers [29] theory to remove the prediction outliers results.


## 2.2.4. Support Vector Regression

The support vector regression (SVR) is introduced by Gunn [31] and proposed by Suykens and Vandewalle [32] as tool for nonlinear modeling of different behaviors of engineering application [11, 12, 33]. The SVR performance with small samples is more popular than traditional statistical methods to model the nonlinear of dynamic behavior [12]. This study investigates the use of VSR models for the settlement prediction of pile-raft foundations. The SVR model maps input data into a higher dimensional feature space via a nonlinear mapping, and then a linear regression problem is obtained and solved in this space [33]. The SVR is described using ε-insensitive loss function as follows [31]:(5)Lεy=0foryx-y<εyx-y-εotherwisewherey(x) and y are predicted and observed vectors and Lεy indicates that it does not penalize errors below ε. The nonlinear SVR solution is given by(6)maxα,α∗⁡Wα,α∗=maxα,α∗⁡∑i=1Nαi∗yi-ε-αiyi+ε-0.5∑i=1N∑j=1Nαi∗-αiαj∗-αjKxi,xjwith constraints(7)0≤α,α≤C,i=1,2,…N∑i=1Nαi∗-αiwhere K is a kernel function and C is a regularized constant determining the tradeoff between the training error and model fitness. Solving (6) with the constraints presented in (7) to calculate the Lagrange multiplier (α) and the regression function, we get the following:(8)yx=∑i=1Nαi-αi∗Kxi,x+bwhere b is a bias term. Based on the nature of quadratic programming, only a number of coefficients among αiandαi∗ will be nonzero, and the data points associated with them refer to support vector [33]. Many kernel functions can be used in this case; the Gaussian kernel function is the most popular function utilized to predict the nonlinear of dynamic behavior [33]; in this study the linear, polynomial, and Gaussian functions are evaluated.


## 2.2.5. Model Design and Performance Evaluation

Based on Ghorbani and Niavol [5] results and the developed model, the five parameters (N, d, s/d, l/d, and p) are used for the settlement prediction of pile-raft foundations. One hundred and forty-four data samples are utilized in this study. The data set is divided into training 75% (107 samples) and testing 25% (37 samples) subsets. Table 2 shows the statistical analysis for the training and testing subsets. It is clear that the input data sets have the same statistical parameters evaluation, approximately, while the settlement parameters for the training and testing have some changes and this may be caused by some extrapolation in the predicting values.Table 2
Statistical analysis of the training and testing data samples.
Training N d (m) s/d l/d P (kPa) S (cm) Mean 16.48 0.40 5.00 23.03 74.86 6.65  Max 25.00 0.50 6.00 32.00 90.00 12.31  Min 9.00 0.30 4.00 16.00 60.00 2.83  sd 5.98 0.10 0.82 5.96 15.07 2.31  Testing N d (m) s/d l/d P (kPa) S (cm)  Mean 17.22 0.40 5.00 22.92 75.41 6.72  Max 25.00 0.50 6.00 32.00 90.00 11.35  Min 9.00 0.30 4.00 16.00 60.00 3.62  sd 8.11 0.10 0.82 5.94 15.20 1.91Figure4 shows the four stages of the prediction process for the four models. First, the input and output data are prepared and processed by removing the outlier’s data; second, the four models parameters are developed using training data set and evaluated using testing data set. In the third stage, the four models results are compared to select the best model that can be used to predict the settlement; then the selected model results are compared with the Ghorbani and Niavol [5] results.Figure 4
Models processing.To evaluate the models, three statistical measures are utilized: coefficient of determination (R2), route mean square error (RMSE), and mean absolute error (MAE):(9)R2=1-∑i=1Nyi-fi2∑i=1Nyi-y-2(10)RMSE=0.5∑i=1Nyi-fi2(11)MAE=1N∑i=1Nyi-fiwherey and f are the actual and predicted settlements values, y- is the mean value for the actual values, and N is the number of observations.


## 3. Results and Discussions

For the MARS model, the number of BFs should be selected first. The mean square error (MSE) and GCM are calculated to estimate the BFs numbers for the training data set. Figure5(a) shows the MSE and GCM for the 18 BFs. From this figure, the 14 BFs are selected to develop the settlement model, where MSE and GCM almost converge and MSE changes slightly. Moreover, Figure 5(b) presents the BFs coefficients’ values which can be used in (1) for settlement prediction based on the BFs equations presented in Table 3. The values of the model’s evaluation parameters R2, RMSE, and MAE are 0.99, 0.18 cm, and 0.13 cm, respectively; these results indicate that the designed model can be used to predict the settlements.Table 3
Basis functions and corresponding equations of MARS model for settlement.
BF Equation BF Equation BF1 max ⁡ ( 0 , x 4 - 24 ) BF8 B F 3 ∗ max ⁡ ( 0 , x 4 - 24 ) BF2 max ⁡ ( 0,24 - x 4 ) BF9 B F 3 ∗ max ⁡ ( 0,24 - x 4 ) BF3 max ⁡ ( 0 , x 5 - 60 ) BF10 B F 2 ∗ max ⁡ ( 0 , x 3 - 4 ) BF4 max ⁡ ( 0 , x 3 - 5 ) BF11 B F 3 ∗ max ⁡ ( 0 , x 2 - 0.3 ) BF5 max ⁡ ( 0,5 - x 3 ) BF12 max ⁡ ( 0,20 - x 4 ) BF6 max ⁡ ( 0 , x 1 - 16 ) BF13 max ⁡ ( 0 , x 4 - 20 ) ∗ max ⁡ ( 0,16 - x 1 ) BF7 max ⁡ ( 0,16 - x 1 ) BF14 B F 3 ∗ max ⁡ ( 0 , x 1 - 16 )Figure 5
BFs number and coefficient for the settlement prediction: (a) BFs numbers, (b) BFs coefficients.
(a) (b)To design the M5Tree model, the regression tree and model tree, combined regression with linear regression function at the leaves, are utilized. In this study, the two models of tree are examined. For the model tree, the number of trees should be selected first to improve the nonlinearity performance of the settlement prediction. Figure6 shows the MSE for the prediction model of training data with 200 trees. It is seen that 24 trees are enough to estimate lower MSE (1.38 cm) for the prediction model. The regression tree, as well, is evaluated and it was found that the MSE for the prediction model is 0.57 cm with 35 rules. From these results, it can be observed that the regression tree is better to predict the settlement. The designed regression tree model is presented in Table 4; the R2, RMSE, and MAE for this model are 0.97, 0.41 cm, and 0.27 cm, respectively, during the training stage. Thus, this model can be used to estimate settlement values.Table 4
Regression tree obtained from M5Tree for settlement modeling.
if x4 <= 22if x5 <= 75 if x3 <= 4.5 y = 7.85 (8) else if x1 <= 12.5 if x4 <= 18 y = 5.9533 (3) else y = 5.0467 (3) else if x4 <= 18 y = 7.2633 (6) else if x1 <= 20.5 if x3 <= 5.5 y = 6.06 (2) else y = 5.66 (2) else y = 6.625 (2) else if x3 <= 4.5 if x4 <= 18 y = 11.677 (5) elseif x2 <= 0.4 y = 9.4867 (3) else y = 10.695 (2) else if x4 <= 18 if x1 <= 12.5 y = 8.8236 (2) else if x3 <= 5.5 y = 10.335 (4) else y = 9.53 (3) else if x2 <= 0.4 y = 7.6925 (4) else y = 8.69 (4) else if x5 <= 75 if x4 <= 28 if x3 <= 4.5 y = 5.6125 (4) else if x1 <= 12.5 if x3 <= 5.5 y = 3.965 (2) elsey = 3.695 (2) else if x1 <= 20.5 if x3 <= 5.5 y = 4.585 (2) else y = 4.335 (2) else y = 5.16 (2) else if x1 <= 12.5 y = 3.148 (5) else if x3 <= 4.5 y = 4.39 (3) else if x1 <= 20.5 if x3 <= 5.5 y = 3.61 (2) else y = 3.39 (2) else y = 4.135 (2) elseif x4 <= 28 if x3 <= 4.5 if x1 <= 20.5 y = 7.36 (3) else y = 8.245 (2) else if x1 <= 20.5 if x2 <= 0.4 y = 5.7625 (4) else y = 6.27 (2) else y = 6.94 (2) else if x3 <= 4.5 if x1 <= 20.5 y = 5.7367 (3) else y = 6.315 (2) else if x2 <= 0.4 y = 4.4675 (4) else y = 5.36 (4)Figure 6
M5Tree model tree selection.In the GRNN model design, the spread orσ value is an important factor in the model design [9]. Figure 7 shows the RMSE of different spread values selected for the training data set. The lowest RMSE value resulted at a spread value of 0.05 and, accordingly, this value is used to design the GRNN model for settlement prediction. The performance parameters of this model, R2, RMSE, and MAE, are 0.99, 4.67e-6 cm, and 2.89e-6 cm, respectively. Based on these results, this model can be used to estimate the settlements of the foundation with higher accuracy than other models.Figure 7
The effect of spread constant on GRNN model.Finally, the design of the SVR model for foundations settlement prediction depends mainly on theα vector and kernel function, (8). In this study the linear, Gaussian, and cubic poly kernels functions are assessed. The RMSE for the Gaussian function is 0.09 cm, while the other functions errors are high (RMSE=NAN value); however, the Gaussian kernel function is used in this study. Figure 8 presents the α vector calculated for the Gaussian kernel function with the model b value equal to −2.08e+6. The R2, RMSE, and MAE for the training data set are 0.99, 0.09 cm, and 0.09 cm, respectively. The results for training data show that this model is accurate to detect the settlements of pile-raft foundations.Figure 8
Lagrange multiplier vector for the SVR model.As per the results presented, the four designed models outperformed the neural network and the genetic algorithm for the Ghorbani and Niavol [5] study in the training stage. The R2 values for the neural network and genetic algorithm are reported as 0.96 and 0.97, respectively [5]. Therefore, the testing stage is used to detect the behavior of the four models with small number of data sets. Figure 9 presents the relationship between the training and testing data for the measured and predicted settlements. In addition, Table 5 presents the performance of four models in both sets.Table 5
Statistical performance of the designed settlement prediction models.
Model Training Testing R2 RMSE (cm) MAE (cm) R2 RMSE (cm) MAE (cm) MARS 0.993 0.179 0.133 0.983 0.245 0.195  M5Tree 0.968 0.413 0.275 0.869 0.681 0.529  GRNN 0.999 4.67e-06 2.89e-06 0.962 0.366 0.316  SVR 0.995 0.096 0.095 0.994 0.148 0.131Figure 9
Measured and predicted settlement for the (a) MARS, (b) M5Tree, (c) GRNN, and (d) SVR models.
(a) (b) (c) (d)The performance of the MARS model is almost constant during both the training and testing stages. The slopes of the linear regression for the training and testing cases are found to be 0.99, and 0.94, respectively. In addition, the R2, RMSE, and MAE are 0.01, 0.066 cm, and 0.062 cm, respectively (Table 5). The worst prediction is reported with the M5Tree model with the R2, RMSE, and MAE being 0.10, 0.268 cm, and 0.254 cm, respectively. The slopes of the linear regression for the training and testing sets are found to be 0.97 and 0.93, respectively. The GRNN model, on the other hand, outperforms the M5Tree with small number of data sets. The slopes of linear fitting of the measured and predicted values for the training and testing phases are 1.0 and 0.95, respectively. In addition, the R2, RMSE, and MAE are reported to be 0.03, 0.366 cm, and 0.316 cm, respectively. The SVR model outperformed the above three models in the training and testing data sets with The R2, RMSE, and MAE values being 0.001, 0.052 cm, and 0.036 cm, respectively. Also, the slopes of linear fitting for this model for the predicted and d values in the training and testing phases are almost equal (0.99). From these results, the maximum changes for the RMSE and MAE are reported with the M5Tree and GRNN models. Based on the presented discussion, it is noted that the MARS and SVR models outperformed the M5Tree and GRNN models in the testing stage. Accordingly, the MARS and SVR models are used to detect the settlements of the pile-raft foundations under static and dynamic loads.Finally, the results of the developed models are compared with Ghorbani and Niavol [5] models. Ghorbani and Niavol [5] used feed-forward back-propagation neural network and genetic algorithm models to predict foundations settlement and compared the results of both models. Their results show that the R2 values for the neural network and genetic algorithm are 0.97 and 0.973, respectively. In the current study, the MARS and SVR models for testing data sets outperformed the neural network and genetic algorithm models. As such, the developed MARS and SVR models can be used for foundations settlement prediction under static and dynamic loads effects.Figure10 presents the MARS and VSR prediction values and compared them with the extracted simulation settlements values for the training and testing data sets. From this figure, it can be seen that the correlation between measured and predicted values is high. Finally, from Table 5, Figure 10, Qin et al. [12] conclusions, and the above discussions, it can be concluded that the SVR model is better for predicting foundations settlements with low number of available data set, while we recommend using the MARS model with high number of data sets.Figure 10
Measured and predicted values for the SVR and MARS models for the (a) training and (b) testing sets.
(a) (b)


## 4. Conclusions

This study investigates the use of soft computing models based on heuristic regression methods to predict the settlements of pile-raft foundations. Four models, namely, MARS, M5Tree, GRNN, and SVR, are introduced, designed, evaluated, and compared with Ghorbani and Niavol [5] models. The settlements for a simulation model of different pile-raft foundations under static and dynamic loads are used in this study. Based on the obtained results, the following conclusions are drawn.The heuristic regression tools can be used effectively for predicting the settlements of pile-raft foundations under static and dynamic loads. The evaluation of the four models in the training stage shows that the GRNN model outperformed the other models, with the M5Tree model performing the worst. Comparing the results of the four models with previous study shows that the four models can be used to predict the foundations settlement.The results in the testing stage show a change in the performance of the four designed models. The MARS and SVR models performed better than the GRNN model. Moreover, the SVR model outperformed all the other models and can be used to accurately predict foundations settlement. The results, also, show that the SVR performance is high when low numbers of data sets are used while it is recommended to use the MARS model with higher number of data sets.



---

*Source: 3425461-2018-06-14.xml*