# Evolution of Deep Neural Network Architecture Using Particle Swarm Optimization to Improve the Performance in Determining the Friction Angle of Soil

Authors: Tuan Anh Pham; Van Quan Tran; Huong-Lan Thi Vu
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

This study focuses on the use of deep neural network (DNN) to predict the soil friction angle, one of the crucial parameters in geotechnical design. Besides, particle swarm optimization (PSO) algorithm was used to improve the performance of DNN by selecting the best structural DNN parameters, namely, the optimal numbers of hidden layers and neurons in each hidden layer. For this aim, a database containing 245 laboratory tests collected from a project in Ho Chi Minh city, Vietnam, was used for the development of the proposed hybrid PSO-DNN model, including seven input factors (soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of soil layer, and bottom elevation of soil layer) and the friction angle was considered as the target. The data set was divided into three parts, namely, the training, validation, and testing sets for the construction, validation, and testing phases of the model. Various quality assessment criteria, namely, the coefficient of determination (R2), mean absolute error (MAE), and root mean square error (RMSE), were used to estimate the performance of PSO-DNN models. The PSO algorithm showed a remarkable ability to find out an optimal DNN architecture for the prediction process. The results showed that the PSO-DNN model using 10 hidden layers outperformed the DNN model, in which the average correlation improvement increased R2 by 1.83%, MAE by 5.94%, and RMSE by 8.58%. Besides, a global sensitivity analysis technique was used to detect the most important inputs, and it showed that, among the seven input variables, the elevation of top and bottom of soil played an important role in predicting the friction angle of soil.


---

## Body

## 1. Introduction

In geotechnical engineering, the internal friction angle of soil is one of the most important parameters [1]. This index is mentioned in many studies in the field of geology, for instance, in the investigation of the foundation bearing capacity [2–7], soil stability [3, 8–10], limit loads [11], and slope stability [9, 12–14]. In slope stability or retaining walls-related problems, the friction angle was proven to affect the soil slope stability significantly, as an increase of the friction angle decreases the soil consolidation [3]. Similarly, the retaining wall’s reaction is also significantly affected by the internal friction angle [8]. In another study, You Mo et al. [11] pointed out a positive correlation between friction angle and critical load. The above points showed that the friction angle is a vital soil parameter [1], and more in-depth research of the latter is required. However, the friction angle is only mentioned indirectly in related geological studies [2–4, 8, 9, 11], and there are limited researches on this index.The value of the internal friction angle depends on many soil parameters, such as grain size, particle shape, unit weight, specific gravity, and soil moisture [15, 16]. Two main approaches were proposed for estimating the friction angle of soil: experimental and theoretical approaches. Concerning the experimental one, the internal friction angle can be obtained from the direct shear test [1, 17–19] and the triaxial shear test [4, 19–22]. However, these experiment testing methods possessed some disadvantages, as they are time-consuming and required relatively expensive equipment [23–25]. Besides, theoretical and numerical methods were also applied to determine the friction angle of soil. As an example, Mo et al. [11] proposed an equation for the friction angle of the soil and confirmed the validity of the theoretical method.Notwithstanding the foregoing, the theoretical method’s limitation is that it is necessary to be based on some assumptions, in this case, the ideal rigid-plastic slip-line field theory. Other studies proposed alternative methods to evaluate the internal friction angle of soil, based on the standard penetration test (SPT) values [26–29] and cone penetration test (CPT) results [30, 31]. Those methods provided several empirical formulas that could quickly predict the soil friction angle. However, these formulas contained only one or several main input variables, which could reduce the prediction accuracy in many cases.In the last two decades, based on the development of computer science, numerous studies have proposed a novel approach for surveying and assessing geological issues, namely, the artificial intelligence (AI) approach [10, 25, 32, 33]. In Mikaeil et al.’s work [32], the authors have used the genetic algorithm (GA) to study three crucial physical and mechanical characteristics of soil, in which the internal friction angle is an important parameter. Additionally, an adaptive neuro-fuzzy inference system (ANFIS) has been used by Murlidhar et al. [25] to study the shear strength of rock based on the internal friction angle. Das and Basudhar [34] and Al-Hamed et al. [35] have developed ANN models to predict the internal friction angle of soil. Besides, Pham et al. [36] have developed a hybrid model using random forest and particle swarm optimization for estimation of undrained shear strength of soil. The obtained results showed the effectiveness and reliability of AI approach, reflected by the high R2 values achieved [10, 25, 32, 33]. Besides, Deep Neural Network (DNN) is also another powerful and efficient algorithm of AI. Nowadays, DNN has become popular and widely used to solve practical engineering problems and provide reliable results [37–40]. The problem with such an ML approach is that it has a large number of critical hyperparameters, making it difficult to find an optimal model architecture. Among various optimization algorithms, Particle Swarm Optimization (PSO) is an efficient, robust, and straightforward algorithm mainly used to solve problems that are difficult to find an exact mathematical model [41–43].Therefore, the main objective of this study is to develop a PSO-DNN hybrid model, which can be self-developed to find the best architectural model to predict the internal friction angle of the soil, including the number of hidden layers and number of neurons in each hidden layer. Seven input factors that might affect the prediction of friction angle of soil were considered: the soil state, standard penetration test value, unit weight of soil, void ratio, thickness of soil layer, top elevation of the soil layer, and bottom elevation of the soil layer. For this aim, a data set consisting of 245 soil samples was collected from some drill holes in Ho Chi Minh City, Vietnam. The database was then divided into the training (60% of the data set), validation (20% of the data set), and testing (20% of the data set), related to the training, validation, and testing phases of the hybrid model. In addition, a global sensitivity analysis method using Monte Carlo simulation was conducted to find out the most important parameter that affects the prediction of friction angle of soil.


## 2. Data Collection and Preparation

### 2.1. Experimental Measurement of Friction Angle of Soil

In this work, the authors conducted 245 identified different soil samples in Ho Chi Minh City, Vietnam (Figure1), and the results are summarized in Table 1. To determine the characteristics of the soil properties, we use the method of determination of shear laboratory resistance in a shear box apparatus [44, 45]. The test samples with the original structure and natural moisture were prepared by cutting from the original soil samples into blocks, which were taken into the ring knives by the method of determining the volume by the ring knife [46]. The underside and top of the soil sample should be leveled with the circular knife's edge and placed with dampened paper first. For fast cutting without draining, the absorbent paper must be replaced with tracing paper (or thin plastic) [46]. Simultaneously with cutting soil samples, it is necessary to take soil to determine moisture content.Figure 1
Experimental location (∗). Source (∗): CIA Map.Table 1
Inputs and output of the present study.
N°SN30GeHZ1Z2(ϕ)Unit(kN/m3)(m)(m)(m)(o)13818.950.8463.6004.3007.90016.98324819.330.7823.4001.1004.50020.042341719.470.76514.0001.20015.20022.32843818.90.8423.1003.2006.30015.600..................2433518.680.9306.6002.0008.6008.87024461620.630.5605.4004.0009.40016.42024564420.410.58413.10040.00053.10016.902Min2216.20.4160.8000.0002.0004.23Average4.64914.62919.240.7846.30513.36919.67316.648Max98521.571.39433.00077.32079.32033.4SD1.88210.9400.9480.1945.56014.41415.5906.953With sandy soil, the sample was prepared by pouring dry sand onto a cutter installed in a hard bottom box with many small holes. Then, take a sample into a cylindrical ring knife with two ends, place the sample in the cutting box, and compress the sample under pressureσ. If soil does not preconsolidate, cut samples immediately, cut quickly, and cut samples at a speed of 1 mm/minute until the sample is damaged. If the soil consolidates under pressure σ, maintain that compressive force until the level of consolidation is met, and then proceed as normal. It is required to cut slowly at a cutting rate of 0.01 mm/min (or slower) until the sample breaks. The destructive force is the maximum value read on the strain gauges. The vertical compression pressure was applied at four levels: 50 kPa, 100 kPa, 200 kPa, and 300 kPa, respectively. For each level of compressive pressure, the corresponding shear strength of soil was recorded. Draw the shear strength according to the vertical pressure, thereby determining the cohesion of soil C, tgϕ and calculating the internal friction angle (ϕ).


### 2.2. Data Preparation

To predict the internal friction angle of the soil, the inputs need to be carefully considered. First, the water content was not included in this study, as it is believed that the effect was taken into account in the number of SPT blows [47]. Besides, many studies have been successful in finding the relationship between the results of in situ experiments (CPT and SPT) and the Plastic and Liquid Limited humidity [48–51]. Since the SPT is one of the most commonly used tests in practice to indicate the soil's ability to withstand compression and in situ shear, it is believed to relate to the factors that characterize the ability of the shear strength of the soil. Many studies have shown the relationship between the in situ experiment results and the internal friction angle of the soil [26–31]. Second, this study aims to provide a tool to quickly analyze the internal friction angle of the soil based on basic parameters that can be quickly and simply determined instead of parameters that take time and cost for determination. Therefore, this study selected SPT value (N30) of soil as the main feature, and some other indicators such as soil state (S), unit weight of soil (G), void ratio (e), thickness of soil layer (H), top elevation of soil layer (Z1), and bottom elevation of soil layer (Z2) are inputs to predict the internal friction angle of soil. The internal friction angle of soil (ϕ) is the single output variable.One of the most popular methods to determine the internal friction angle of soil (ϕ) is the direct shear test method [44, 45]. There are three soil direct cutting modes: UU mode (this mode is fast cutting-no cohesion, no drainage), CU mode (fast cutting, cohesive-no drainage), and CD mode (slow cutting, consolidation-drainage; this mode usually applies to sandy soils) [1]. Firstly, the UU cutting mode is suitable for fast construction and difficult drainage, and the soil samples in the present study were all under this type of condition. Secondly, the UU cutting mode is quick test, taking under 30 minutes to perform, while the CU and CD modes take longer to execute and can take weeks or even months to complete. Besides, the shear strength parameters given by the UU cutting mode are generally safer than those in the other two modes. Therefore, in this study, the results of the UU cutting mode were applied to determine the friction angle of all soil samples. The meaning of the soil type state number is explained in Table 2. Besides, the parameters’ meanings are illustrated in Figure 2, and the histograms of all the input and output variables are shown in Figure 3.Table 2
Soil state number explanation.
Soil typeSoil state numberSoil type-stateClay1Very soft clay2Soft clay3Firm clay4Stiff clay5Very stiff clay6Hard claySand7Loose sandy8Medium sandy9Dense sandyFigure 2
Diagram for parameters.Figure 3
Histograms of the data set in this study. (a) S. (b) N30. (c) G. (d) H. (e)Z1. (f) Z2. (g) ϕ (°).
(a)(b)(c)(d)(e)(f)(g)The data set containing 245 samples was statistically introduced and summarized in Table1, including several soil samples, min, max, average, and standard deviation of the input and output variables. It can be seen that, among 245 soil samples, 46 samples are sandy soil and the rest are clayey soil. Soil state number (S) ranged from 2 to 9, corresponding to soft clay to dense sandy soil. The minimum depth of the soil samples is 2.0 m, and the maximum depth is 79.32 m. The standard penetration test values (N30) were from 2 to 85, where 2 corresponds to the soft clay soil and 85 corresponds to the hard clay soil. The unit weight of soil (G) ranged from 16.2 (kN/m3) to 21.57 (kN/m3) in accordance with the clay soil to sandy soil layers. The void ratio (e) varied from 0.416 to 2.194, corresponding from medium sandy soil to soft clay soil. The internal friction angles of soil ranged from 4.23 (degrees) to 33.4 (degrees), related to the shear strength of the soil increasing from soft clay to medium and dense sandy soil. These input data results are consistent with the properties of the soil layers. An example of 50 data samples is given in Table 3.Table 3
An example of data set of the present study.
N°SN30GeHZ1Z2ϕUnitkN/m3Mmm(degree)13818.950.8463.64.37.916.98324819.330.7823.41.14.520.042341719.470.765141.215.222.32843818.900.8423.13.26.315.6571018.820.7904.3711.328.1863819.700.68531.14.11773819.700.68531.14.11784919.440.7963.10.53.618.7393718.610.8874.73.78.414.85103319.100.8062.312.71515.295113319.100.8062.312.71515.2951285021.280.4311111233.4132417.801.1930.83.248.02142417.801.1930.83.248.021541519.180.7813.510.51418.2162217.101.2721.53.95.47.031782420.120.5439.525.935.432.81841320.100.58526.48.63524.171941320.100.58526.48.63524.172081520.900.4576.19.415.532.57213418.980.8294.526.514.533223418.980.8294.526.514.5332382419.700.6902.728.831.529.3244919.440.7525.30.7618.12582220.200.5716.933.14031.032641920.200.56430.156.736.8524.22782620.210.5736.433.64030.8552882620.210.5736.433.64030.855293617.781.07015.54.52013.8543081220.900.4605556031.753165020.550.5715.668.47425.23281420.670.4775.73338.731.6233352019.380.7445.831.537.320.7443481819.520.6743.452730.4529.08353718.840.8799.31.210.514.5723681220.700.50047.211.230.725373418.850.85141.55.513.742383418.850.85141.55.513.742393718.940.8457.7714.714.4403718.940.8457.7714.714.44134219.400.714524.629.6154251819.670.7056.70.3720.8834351819.670.7056.70.3720.8834431019.200.792210.512.513.874531019.200.792210.512.513.874641418.300.8305.42.47.816.4334781320.260.57013.46.62031.2054842219.430.7471.21011.216.7664941418.850.7922.25.8816.3155081219.150.7353576027.505In this study, we divided the data into three data sets, in which the training set was used to build the model, the validation set was used to give an estimate of model skill while tuning the model’s hyperparameters, and the testing set was used to estimate of the skill of the final tuned models to choose the best model. The test set was fixed at 20% of the total data set, hidden during training, and used only to evaluate the performance of the final model after the hyperparameters tuning process.


## 2.1. Experimental Measurement of Friction Angle of Soil

In this work, the authors conducted 245 identified different soil samples in Ho Chi Minh City, Vietnam (Figure1), and the results are summarized in Table 1. To determine the characteristics of the soil properties, we use the method of determination of shear laboratory resistance in a shear box apparatus [44, 45]. The test samples with the original structure and natural moisture were prepared by cutting from the original soil samples into blocks, which were taken into the ring knives by the method of determining the volume by the ring knife [46]. The underside and top of the soil sample should be leveled with the circular knife's edge and placed with dampened paper first. For fast cutting without draining, the absorbent paper must be replaced with tracing paper (or thin plastic) [46]. Simultaneously with cutting soil samples, it is necessary to take soil to determine moisture content.Figure 1
Experimental location (∗). Source (∗): CIA Map.Table 1
Inputs and output of the present study.
N°SN30GeHZ1Z2(ϕ)Unit(kN/m3)(m)(m)(m)(o)13818.950.8463.6004.3007.90016.98324819.330.7823.4001.1004.50020.042341719.470.76514.0001.20015.20022.32843818.90.8423.1003.2006.30015.600..................2433518.680.9306.6002.0008.6008.87024461620.630.5605.4004.0009.40016.42024564420.410.58413.10040.00053.10016.902Min2216.20.4160.8000.0002.0004.23Average4.64914.62919.240.7846.30513.36919.67316.648Max98521.571.39433.00077.32079.32033.4SD1.88210.9400.9480.1945.56014.41415.5906.953With sandy soil, the sample was prepared by pouring dry sand onto a cutter installed in a hard bottom box with many small holes. Then, take a sample into a cylindrical ring knife with two ends, place the sample in the cutting box, and compress the sample under pressureσ. If soil does not preconsolidate, cut samples immediately, cut quickly, and cut samples at a speed of 1 mm/minute until the sample is damaged. If the soil consolidates under pressure σ, maintain that compressive force until the level of consolidation is met, and then proceed as normal. It is required to cut slowly at a cutting rate of 0.01 mm/min (or slower) until the sample breaks. The destructive force is the maximum value read on the strain gauges. The vertical compression pressure was applied at four levels: 50 kPa, 100 kPa, 200 kPa, and 300 kPa, respectively. For each level of compressive pressure, the corresponding shear strength of soil was recorded. Draw the shear strength according to the vertical pressure, thereby determining the cohesion of soil C, tgϕ and calculating the internal friction angle (ϕ).


## 2.2. Data Preparation

To predict the internal friction angle of the soil, the inputs need to be carefully considered. First, the water content was not included in this study, as it is believed that the effect was taken into account in the number of SPT blows [47]. Besides, many studies have been successful in finding the relationship between the results of in situ experiments (CPT and SPT) and the Plastic and Liquid Limited humidity [48–51]. Since the SPT is one of the most commonly used tests in practice to indicate the soil's ability to withstand compression and in situ shear, it is believed to relate to the factors that characterize the ability of the shear strength of the soil. Many studies have shown the relationship between the in situ experiment results and the internal friction angle of the soil [26–31]. Second, this study aims to provide a tool to quickly analyze the internal friction angle of the soil based on basic parameters that can be quickly and simply determined instead of parameters that take time and cost for determination. Therefore, this study selected SPT value (N30) of soil as the main feature, and some other indicators such as soil state (S), unit weight of soil (G), void ratio (e), thickness of soil layer (H), top elevation of soil layer (Z1), and bottom elevation of soil layer (Z2) are inputs to predict the internal friction angle of soil. The internal friction angle of soil (ϕ) is the single output variable.One of the most popular methods to determine the internal friction angle of soil (ϕ) is the direct shear test method [44, 45]. There are three soil direct cutting modes: UU mode (this mode is fast cutting-no cohesion, no drainage), CU mode (fast cutting, cohesive-no drainage), and CD mode (slow cutting, consolidation-drainage; this mode usually applies to sandy soils) [1]. Firstly, the UU cutting mode is suitable for fast construction and difficult drainage, and the soil samples in the present study were all under this type of condition. Secondly, the UU cutting mode is quick test, taking under 30 minutes to perform, while the CU and CD modes take longer to execute and can take weeks or even months to complete. Besides, the shear strength parameters given by the UU cutting mode are generally safer than those in the other two modes. Therefore, in this study, the results of the UU cutting mode were applied to determine the friction angle of all soil samples. The meaning of the soil type state number is explained in Table 2. Besides, the parameters’ meanings are illustrated in Figure 2, and the histograms of all the input and output variables are shown in Figure 3.Table 2
Soil state number explanation.
Soil typeSoil state numberSoil type-stateClay1Very soft clay2Soft clay3Firm clay4Stiff clay5Very stiff clay6Hard claySand7Loose sandy8Medium sandy9Dense sandyFigure 2
Diagram for parameters.Figure 3
Histograms of the data set in this study. (a) S. (b) N30. (c) G. (d) H. (e)Z1. (f) Z2. (g) ϕ (°).
(a)(b)(c)(d)(e)(f)(g)The data set containing 245 samples was statistically introduced and summarized in Table1, including several soil samples, min, max, average, and standard deviation of the input and output variables. It can be seen that, among 245 soil samples, 46 samples are sandy soil and the rest are clayey soil. Soil state number (S) ranged from 2 to 9, corresponding to soft clay to dense sandy soil. The minimum depth of the soil samples is 2.0 m, and the maximum depth is 79.32 m. The standard penetration test values (N30) were from 2 to 85, where 2 corresponds to the soft clay soil and 85 corresponds to the hard clay soil. The unit weight of soil (G) ranged from 16.2 (kN/m3) to 21.57 (kN/m3) in accordance with the clay soil to sandy soil layers. The void ratio (e) varied from 0.416 to 2.194, corresponding from medium sandy soil to soft clay soil. The internal friction angles of soil ranged from 4.23 (degrees) to 33.4 (degrees), related to the shear strength of the soil increasing from soft clay to medium and dense sandy soil. These input data results are consistent with the properties of the soil layers. An example of 50 data samples is given in Table 3.Table 3
An example of data set of the present study.
N°SN30GeHZ1Z2ϕUnitkN/m3Mmm(degree)13818.950.8463.64.37.916.98324819.330.7823.41.14.520.042341719.470.765141.215.222.32843818.900.8423.13.26.315.6571018.820.7904.3711.328.1863819.700.68531.14.11773819.700.68531.14.11784919.440.7963.10.53.618.7393718.610.8874.73.78.414.85103319.100.8062.312.71515.295113319.100.8062.312.71515.2951285021.280.4311111233.4132417.801.1930.83.248.02142417.801.1930.83.248.021541519.180.7813.510.51418.2162217.101.2721.53.95.47.031782420.120.5439.525.935.432.81841320.100.58526.48.63524.171941320.100.58526.48.63524.172081520.900.4576.19.415.532.57213418.980.8294.526.514.533223418.980.8294.526.514.5332382419.700.6902.728.831.529.3244919.440.7525.30.7618.12582220.200.5716.933.14031.032641920.200.56430.156.736.8524.22782620.210.5736.433.64030.8552882620.210.5736.433.64030.855293617.781.07015.54.52013.8543081220.900.4605556031.753165020.550.5715.668.47425.23281420.670.4775.73338.731.6233352019.380.7445.831.537.320.7443481819.520.6743.452730.4529.08353718.840.8799.31.210.514.5723681220.700.50047.211.230.725373418.850.85141.55.513.742383418.850.85141.55.513.742393718.940.8457.7714.714.4403718.940.8457.7714.714.44134219.400.714524.629.6154251819.670.7056.70.3720.8834351819.670.7056.70.3720.8834431019.200.792210.512.513.874531019.200.792210.512.513.874641418.300.8305.42.47.816.4334781320.260.57013.46.62031.2054842219.430.7471.21011.216.7664941418.850.7922.25.8816.3155081219.150.7353576027.505In this study, we divided the data into three data sets, in which the training set was used to build the model, the validation set was used to give an estimate of model skill while tuning the model’s hyperparameters, and the testing set was used to estimate of the skill of the final tuned models to choose the best model. The test set was fixed at 20% of the total data set, hidden during training, and used only to evaluate the performance of the final model after the hyperparameters tuning process.


## 3. Machine Learning Methods

### 3.1. Particle Swarm Optimization

Particle Swarm Optimization (PSO) was presented by Kennedy and Eberhart [52]. It became prevalent because it is a continuous optimization process type and allows an analysis of multiple targets. For a continuous search of the best solution, the method used is to move the positions of particles at a given velocity calculated in each iteration. Each movement of particles is influenced by its best position and the best position in the entire search space (Figure 4). This is expected to move the swarm to the best position. The effectiveness of the solution is assessed through a fitness function. In addition, PSO does not use the slope of the problem to be optimized, which means that PSO does not require optimization issues to be differentiable like standard optimization methods such as gradient descent and quasi-newton methods. PSO is a powerful technique that has been widely used for optimization issues in many fields in general and geotechnical engineering in particular [53, 54].Figure 4
Particle movement by the swarm direction.The pseudocode of the algorithm is presented as follows. (Algorithm1)Algorithm 1Algorithm 1
FOR each particle i in swarmInitialize parameters:w, w_damp, c1, c2.FOR each dimension jInitialize positionxij randomly within permissible rangeInitialize velocityvij randomly within permissible rangeEnd FOREnd FORIterationk = 1WHILE k < maximum_IterationFOR each particlei in swarmCalculate fitness valueIF fitness value > P_best[i] THENP_best[i] = fitness valueEND IFIF fitness value > G_best THENG_best = fitness valueEND IFEND FORFOR each particle i in swarmFOR each dimension jCalculate new velocity:vij(k + 1) = wvij(k) + c1random (0,1) (P_best[i] − xij[i]) + c2random (0,1) (G_best − xij[i])Update particle positon:xij(k + 1) = xij(k) + vij(k + 1)END FOREND FORw = w.w_dampk = k + 1END WHILEThe parameters in the equation that defines the velocity of the next iteration are as follows:w is an inertial parameter; c₁ and c₂ are the acceleration coefficients; c₁ value gives the importance of individual best solution, and c2 is the importance of global best candidate solution; w_damp is the inertial reduction coefficient, and it helps the swarm movement to quickly converge.


### 3.2. Deep Neural Network

Deep neural network (DNN) is definitely one of the most advanced regression methods. DNN model is analogous to a multistage regression. The main idea is to create a flexible nonlinear statistical model consisting of several layers and neurons in each layer. In DNN, each node in a layer is associated with a certain weight, denoted aswij, with every node in the other layers creating a fully linked neural system [55]. Except for the input layer, each node is a neuron that uses a nonlinear activation function [56]. Thanks to multilayer and nonlinear activation functions, the DNN model could distinguish nonlinear separable data. The DNN structure used in this study is a multilayer perception (MLP), shown in Figure 5. Further details of the DNN model can be found in [57].Figure 5
Illustration of the DNN network used in this study, including 10 input nodes, hidden layers, and one output node.


### 3.3. Particle Swarm Optimization-Deep Neural Network Algorithm

The PSO-DNN hybrid model was created based on the PSO algorithm, in which the fitness function of each particle in the swarm was DNN. The hybrid algorithm is implemented with the number of hidden layers ranging from 2 to 10. For each case of the number of hidden layers, random particles are generated, the length of the particle vector is the number of hidden layers, and thejth dimension of the particle vector is the number of hidden neurons. The swarm architecture was presented in Figure 6, in which k is the number of hidden layers and N is the number of particles in a swarm. The flow chart of the hybrid PSO-DNN model was shown in Figure 7. In this algorithm, the number of hidden layers in the model is incremented from 2 to 10. For each case of the number of hidden layers, the algorithm will automatically look for the optimal model architecture for that number of hidden layers.Figure 6
Description of swarm architecture.Figure 7
Flow chart of PSO-DNN algorithm.


### 3.4. Performance Evaluation

To evaluate the effectiveness of the proposed AI models, three widely used statistical criteria are applied, namely, mean absolute error (MAE), root mean square error (RMSE), and the coefficient of determination (R2). In order to estimate the percentage change of data which can be achieved by prediction, it is common to use the R2 coefficient in regression analysis [58]. To measure the average intensity of error, both MAE and RMSE were used [59]. Studies have concluded that the model will be more accurate when R2 is closer to 1 and MAE and RMSE is closer to 0. These coefficients are determined by the following formulas:(1)MAE=1k∑i=1kyi−y¯i,(2)RMSE=1k∑i=1kyi−y¯i2,(3)R2=1−∑i=1kyi−y¯i2∑i=1kyi−y¯2,where k represents the number of the samples, yi and y¯i were the actual and predicted outputs, respectively, and y¯ was the average value of yi.


## 3.1. Particle Swarm Optimization

Particle Swarm Optimization (PSO) was presented by Kennedy and Eberhart [52]. It became prevalent because it is a continuous optimization process type and allows an analysis of multiple targets. For a continuous search of the best solution, the method used is to move the positions of particles at a given velocity calculated in each iteration. Each movement of particles is influenced by its best position and the best position in the entire search space (Figure 4). This is expected to move the swarm to the best position. The effectiveness of the solution is assessed through a fitness function. In addition, PSO does not use the slope of the problem to be optimized, which means that PSO does not require optimization issues to be differentiable like standard optimization methods such as gradient descent and quasi-newton methods. PSO is a powerful technique that has been widely used for optimization issues in many fields in general and geotechnical engineering in particular [53, 54].Figure 4
Particle movement by the swarm direction.The pseudocode of the algorithm is presented as follows. (Algorithm1)Algorithm 1Algorithm 1
FOR each particle i in swarmInitialize parameters:w, w_damp, c1, c2.FOR each dimension jInitialize positionxij randomly within permissible rangeInitialize velocityvij randomly within permissible rangeEnd FOREnd FORIterationk = 1WHILE k < maximum_IterationFOR each particlei in swarmCalculate fitness valueIF fitness value > P_best[i] THENP_best[i] = fitness valueEND IFIF fitness value > G_best THENG_best = fitness valueEND IFEND FORFOR each particle i in swarmFOR each dimension jCalculate new velocity:vij(k + 1) = wvij(k) + c1random (0,1) (P_best[i] − xij[i]) + c2random (0,1) (G_best − xij[i])Update particle positon:xij(k + 1) = xij(k) + vij(k + 1)END FOREND FORw = w.w_dampk = k + 1END WHILEThe parameters in the equation that defines the velocity of the next iteration are as follows:w is an inertial parameter; c₁ and c₂ are the acceleration coefficients; c₁ value gives the importance of individual best solution, and c2 is the importance of global best candidate solution; w_damp is the inertial reduction coefficient, and it helps the swarm movement to quickly converge.


## 3.2. Deep Neural Network

Deep neural network (DNN) is definitely one of the most advanced regression methods. DNN model is analogous to a multistage regression. The main idea is to create a flexible nonlinear statistical model consisting of several layers and neurons in each layer. In DNN, each node in a layer is associated with a certain weight, denoted aswij, with every node in the other layers creating a fully linked neural system [55]. Except for the input layer, each node is a neuron that uses a nonlinear activation function [56]. Thanks to multilayer and nonlinear activation functions, the DNN model could distinguish nonlinear separable data. The DNN structure used in this study is a multilayer perception (MLP), shown in Figure 5. Further details of the DNN model can be found in [57].Figure 5
Illustration of the DNN network used in this study, including 10 input nodes, hidden layers, and one output node.


## 3.3. Particle Swarm Optimization-Deep Neural Network Algorithm

The PSO-DNN hybrid model was created based on the PSO algorithm, in which the fitness function of each particle in the swarm was DNN. The hybrid algorithm is implemented with the number of hidden layers ranging from 2 to 10. For each case of the number of hidden layers, random particles are generated, the length of the particle vector is the number of hidden layers, and thejth dimension of the particle vector is the number of hidden neurons. The swarm architecture was presented in Figure 6, in which k is the number of hidden layers and N is the number of particles in a swarm. The flow chart of the hybrid PSO-DNN model was shown in Figure 7. In this algorithm, the number of hidden layers in the model is incremented from 2 to 10. For each case of the number of hidden layers, the algorithm will automatically look for the optimal model architecture for that number of hidden layers.Figure 6
Description of swarm architecture.Figure 7
Flow chart of PSO-DNN algorithm.


## 3.4. Performance Evaluation

To evaluate the effectiveness of the proposed AI models, three widely used statistical criteria are applied, namely, mean absolute error (MAE), root mean square error (RMSE), and the coefficient of determination (R2). In order to estimate the percentage change of data which can be achieved by prediction, it is common to use the R2 coefficient in regression analysis [58]. To measure the average intensity of error, both MAE and RMSE were used [59]. Studies have concluded that the model will be more accurate when R2 is closer to 1 and MAE and RMSE is closer to 0. These coefficients are determined by the following formulas:(1)MAE=1k∑i=1kyi−y¯i,(2)RMSE=1k∑i=1kyi−y¯i2,(3)R2=1−∑i=1kyi−y¯i2∑i=1kyi−y¯2,where k represents the number of the samples, yi and y¯i were the actual and predicted outputs, respectively, and y¯ was the average value of yi.


## 4. Results and Discussion

### 4.1. Effect of Training Data Size

Because the testing set size was selected and fixed to 20% of the total data set and hidden from the training process, the training data set size was chosen by dropping from 60% to 20% of the data set, or meaning the size of the validation set increased from 20% to 60% of the remaining data set. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. The initialization parameters of the DNN model used in this section are presented in Table4.Table 4
Initialization parameters of DNN model.
ParametersValue and descriptionNumber of hidden layers2Neurons structure(25, 25)Use early stoppingTrueActivation functionTanhCost functionMSEData usedTraining/validation/testing data setNumber of runs (shuffle input data)300From Figure8, it can be seen that training performance progressively increased with decreasing in the training set. The average R2 value was increased from 0.952 to 0.991 when the training set size was decreased from 60% to 20%. On the other side, the predictions showed less accurate results on the validation set and testing set as the training set size decreases. The average R2 value was decreased from 0.889 to 0.602 on the validation set, while the average R2 value dropped from 0.919 to 0.570 on the testing set. In addition, the standard deviation from the DNN models was increased with decreasing in training set size. To be more specific, the standard deviation was increased from 0.0277 to 0.351 on the testing set. The above results show that the performance of the model got better and became more stable with the most significant training set size case. Aiming to increase the prediction accuracy of the model, the training set size of 60% was selected in this study.Figure 8
Sensitivity analysis of the model using different training set sizes with DNN model.


### 4.2. DNN Structure Optimization

The evolutionary results of PSO-DNN models are evaluated in this section. The initialization parameters of PSO-DNN used in this study are given in Table5. Figure 9 illustrates the evolution of the PSO-DNN model through 30 generations with the number of hidden layers set to values: 2, 4, 6, 8, and 10. A summary of the best predictability of the models is presented in Table 6.Table 5
PSO-DNN initialization parameters.
ParametersExplanationValue and descriptionN_swarmSwarm size50max_iterationNumber of iterations30wiInitial weight0.5w_dampInitial weight damping0.9C1Personal learning coefficient0.4C2Global learning coefficient0.8VarMinNumber of min neurons2VarMaxNumber of max neurons100N_hiddensNumber of hidden layers2, 4, 6, 8, 10VmaxVelocity max0.05∗ (VarMax − VarMin)VminVelocity min−VmaxFitness functionDNNActivation functionTanhEarly stoppingHandling overfittingTrueCost functionR2, MAE, RMSEData usedTraining/validation datasetFigure 9
Neuron structures tuning using the PSO-DNN model with errors criteria: (a) RMSE; (b) MAE; (c)R2.
(a)(b)(c)Table 6
Summary of best prediction capability of PSO-DNN models.
CriteriaNumber of hidden layersBest criteria valueRankBest neuron structureR220.9304(28, 32)40.9262(20, 36, 35, 29)60.9275(17, 44, 52, 52, 38, 28)80.9313(15, 54, 77, 37, 15, 78, 12, 19)100.9381(8, 49, 90, 32, 19, 77, 36, 22, 89, 29)MAE (o)21.4654(9, 19)41.4262(24, 53, 62, 21)61.4443(9, 76, 27, 13, 56, 77)81.5285(32, 25, 51, 74, 36, 35, 22, 76)101.3951(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)RMSE (o)21.9255(7, 22)41.9554(23, 36, 34, 41)61.9213(9, 61, 31, 23, 41, 70)81.8391(17, 55, 47, 54, 66, 39, 41, 28)101.8712(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)Bold values represent the best model for each performance indicator.It can be seen that the DNN model with 10 hidden layers has evolved to bring about better performance; the best generation yielded a correlation ofR2 = 0.938, MAE = 1.395, and RMSE = 1.846. The model with the next best performance is the DNN model with 8 hidden layers which produce accurate intermediate precision (R2 = 0.931, MAE = 1.528, and RMSE = 1.839). The DNN model with 10 hidden layer gives the best performance for R2 and MAE criteria, while the model with 8 hidden layers gives the best result for the RMSE cost function. However, the best neuron structure for each cost function is different and is also shown in Table 6. The evolutions of number of neurons in 10 hidden layers and 8 hidden layers of the best DNN model referring to R2, MAE, and RMSE are presented in Figure 10. It means that the evolution of the model has found three good results; choosing the best model will be done in the next section.Figure 10
Evolution of neurons in each hidden layer using PSO-DNN models: (a) 10 hidden layers,R2; (b) 10 hidden layers, MAE; (c) 8 hidden layers, RMSE (HD∗: hidden layers).
(a)(b)(c)


### 4.3. Predictive Capability of the Models

In this section, the three best models, 10 hidden layers,R2 (10-HD-R2), 10 hidden layers, MAE (10-HD-MAE), and 8 hidden layers, RMSE (8-HD-RMSE) were chosen for comparison with each other. From the point of view of statistical probability, the random factors in the input variables should be considered. Therefore, 300 simulations take into account the randomness of the input variable sequence number. The data used in this section include training, validation, and testing data sets. The criteria for selecting the best model are based on the results of the testing data set. The results are shown in Figure 11 and Tables 7–9 .Figure 11
Performance of PSO-DNN models using criteria:R2 (a), MAE (b), and RMSE (c).
(a)(b)(c)Table 7
Summary of the 300 simulations usingR2 criteria.
ModelData setAverageMinMaxSD10-HD-R2Training0.9460.9150.9700.0083Validation0.8950.7210.9390.0377Testing0.9350.8610.9660.013810-HD-MAETraining0.9000.5860.9530.0477Validation0.8510.4360.9300.0657Testing0.8890.5490.9390.04538-HD-RMSETraining0.9350.8260.950.0156Validation0.9010.6220.9420.0343Testing0.8810.7950.9270.0186Table 8
Summary of the 300 simulations using MAE criteria.
ModelData setAverage (o)Min (o)Max (o)SD10-HD-R2Training1.1880.8831.4230.0899Validation1.7211.3922.5090.2345Testing1.3401.0361.7150.133910-HD-MAETraining1.6361.0502.5600.3031Validation2.1051.4733.4550.3411Testing1.8231.2432.8700.2718-HD-RMSETraining1.2681.1012.1070.1534Validation1.7061.2673.4440.2564Testing1.5231.2382.3380.1923Table 9
Summary of the 300 simulations using RMSE criteria.
ModelData setAverage (o)Min (o)Max (o)SD10-HD-R2Training1.5661.2601.8480.1088Validation2.2741.7823.4980.3395Testing1.771.4062.5920.183510-HD-MAETraining2.0591.4213.1280.3711Validation2.6681.8765.0950.473Testing2.2551.5783.3100.29888-HD-RMSETraining1.7221.4653.5980.2129Validation2.221.6764.8670.3861Testing2.0191.6133.8820.3264The results showed that the predicted results on the training and validation sets gave good accuracy with satisfactory average accuracyR2 ranging from 0.851 to 0.946, MAE ranging from 1.188 to 2.105, and RMSE varying from 1.566 to 2.668, in which it can be seen that, among three good models, the 10-HD-R2 gives the best prediction, which yielded an average correlation of R2 = 0.935, MAE = 1.34, and RMSE = 1.77 on the testing data set. The next best model is 8-HD-RMSE, which produces accurate intermediate precision (R2 = 0.881, MAE = 1.523, and RMSE = 2.019). Moreover, the standard deviation of the 10-HD-R2 model is better than that of the 8-HD-RMSE, where SD = 0.0138, 0.1339, and 0.1835 for R2, MAE, and RMSE criteria compared to SD = 0.0186, 0.1923, and 0.3264 of two models, respectively.Table10 shows the comparison between the DNN model and the PSO-DNN model. The DNN model received initialized parameters as shown in Table 11. The best 10-HD-R2 PSO-DNN model was included for comparison with the proof of the effectiveness of the algorithm. The average error criteria of 300 simulations were compared to confirm the model’s performance. It can be seen that the 10-HD-R2 PSO-DNN model outperformed the DNN model, which improved the average correlation of R2 by 1.83%, MAE by 5.94%, and RMSE by 8.58%.Table 10
Comparison between the DNN model and the PSO-DNN model.
CriteriaDNN modelPSO-DNN modelPerformance improvement (%)R20.9180.9351.83MAE (o)1.4251.3405.94RMSE (o)1.9361.7708.58Table 11
Comparison with previous studies.
AuthorModelParameterNumber of samplesRR2RMSE (o)MAE (o)Khanlari et al. [60]ANNInternal friction angle2000.890.7922.261.92MVR0.890.792.3851.929Mohammadi et al. [61]MLPInternal friction angle1080.9020.8149.2856.137MLR0.7190.5175.0823.954Iyeke et al. [62]ANNInternal friction angle830.8970.8054.774.34Present studyDNNInternal friction angle2450.9580.9181.9361.425PSO-DNN0.9670.9351.771.34Table11 presents some research results on ML applications in determining the internal friction angle of soil. The results of the present study, as well as other studies, show the effectiveness of the ML technique in predicting the internal friction angle of the soil with expected R2 from 0.79 to 0.935 on the testing data set. However, due to the different data sets used and the input variables, it makes no sense to compare these results. A project using different data sets as well as various input variables is needed to provide a general model for the prediction of soil shear parameters.Figure12 shows a visual comparison of test results and predictions based on the friction angle of soil of the DNN and PSO-DNN models. The performances of two ML models have been tested on all three data sets: training, validation, and testing.Figure 12
Measured and predicted values of friction angle of soil using the DNN models: PSO-DNN model for (a) training, (c) validation, and (e) testing data set; DNN model for (b) training, (d) validation, and (f) testing data set.
(a)(b)(c)(d)(e)(f)


### 4.4. Sensitivity Analysis

In this section, a global sensitivity analysis was conducted to evaluate the importance of input parameters for the model using Monte Carlo methods [63]. It is an effective way to investigate a relationship between input and output. The input data set taken from Saltelli’s sampling scheme was used to develop the DNN model [64]. The global sensitivity index is determined by the following formulas:(4)VarY=∑i=1dVari+∑i<jdVarij+∑j<mdVarijm+…+Varij…d,(5)STi=Vari+∑i<jdVarij+∑j<mdVarijm+…+Varij…dVarY,where Var(Y) denotes the total variance of the model output; d denotes the number of input features; Vari denotes the model output variance in response to variation of the ith input variable; Varij denotes the model output variance in response to the simultaneous variation of the ith and the jth input; STi denotes the total sensitivity index.The result of the total sensitivity analysis was shown in Figure13. It can be seen that, among the 7 input variables used to predict the friction angle of soil, the elevation of the soil layer bottom (Z2) was the most important feature, which achieved an average sensitivity index score of 0.625. The topsoil layer elevation (Z1) was the second important variable, confirmed by an average sensitivity index score of 0.603. From the point of view of soil mechanics, the deeper the soil is, the more compact it is, so the depth of the soil sample plays an important role in detecting the friction angle of the soil when its density, state, and many other mechanical properties change according to the depth. The variables H, G, and S were ranked as the third to the fifth important predictors, with an average sensitivity index ranging from 0.203 to 0.05. Other predictor variables included in the model (e, G) had lower than 0.01 in the sensitivity index, indicating that they had not affected the output prediction. It is important to note that the sensitivity analysis in this section is only relevant for the input data set itself and cannot be confirmed with other data sets.Figure 13
Sensitivity analysis using the 10-HD-R2 model.


## 4.1. Effect of Training Data Size

Because the testing set size was selected and fixed to 20% of the total data set and hidden from the training process, the training data set size was chosen by dropping from 60% to 20% of the data set, or meaning the size of the validation set increased from 20% to 60% of the remaining data set. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. For each size case of the training set, 300 simulations were used, taking into account the random shuffle in the training order. The initialization parameters of the DNN model used in this section are presented in Table4.Table 4
Initialization parameters of DNN model.
ParametersValue and descriptionNumber of hidden layers2Neurons structure(25, 25)Use early stoppingTrueActivation functionTanhCost functionMSEData usedTraining/validation/testing data setNumber of runs (shuffle input data)300From Figure8, it can be seen that training performance progressively increased with decreasing in the training set. The average R2 value was increased from 0.952 to 0.991 when the training set size was decreased from 60% to 20%. On the other side, the predictions showed less accurate results on the validation set and testing set as the training set size decreases. The average R2 value was decreased from 0.889 to 0.602 on the validation set, while the average R2 value dropped from 0.919 to 0.570 on the testing set. In addition, the standard deviation from the DNN models was increased with decreasing in training set size. To be more specific, the standard deviation was increased from 0.0277 to 0.351 on the testing set. The above results show that the performance of the model got better and became more stable with the most significant training set size case. Aiming to increase the prediction accuracy of the model, the training set size of 60% was selected in this study.Figure 8
Sensitivity analysis of the model using different training set sizes with DNN model.


## 4.2. DNN Structure Optimization

The evolutionary results of PSO-DNN models are evaluated in this section. The initialization parameters of PSO-DNN used in this study are given in Table5. Figure 9 illustrates the evolution of the PSO-DNN model through 30 generations with the number of hidden layers set to values: 2, 4, 6, 8, and 10. A summary of the best predictability of the models is presented in Table 6.Table 5
PSO-DNN initialization parameters.
ParametersExplanationValue and descriptionN_swarmSwarm size50max_iterationNumber of iterations30wiInitial weight0.5w_dampInitial weight damping0.9C1Personal learning coefficient0.4C2Global learning coefficient0.8VarMinNumber of min neurons2VarMaxNumber of max neurons100N_hiddensNumber of hidden layers2, 4, 6, 8, 10VmaxVelocity max0.05∗ (VarMax − VarMin)VminVelocity min−VmaxFitness functionDNNActivation functionTanhEarly stoppingHandling overfittingTrueCost functionR2, MAE, RMSEData usedTraining/validation datasetFigure 9
Neuron structures tuning using the PSO-DNN model with errors criteria: (a) RMSE; (b) MAE; (c)R2.
(a)(b)(c)Table 6
Summary of best prediction capability of PSO-DNN models.
CriteriaNumber of hidden layersBest criteria valueRankBest neuron structureR220.9304(28, 32)40.9262(20, 36, 35, 29)60.9275(17, 44, 52, 52, 38, 28)80.9313(15, 54, 77, 37, 15, 78, 12, 19)100.9381(8, 49, 90, 32, 19, 77, 36, 22, 89, 29)MAE (o)21.4654(9, 19)41.4262(24, 53, 62, 21)61.4443(9, 76, 27, 13, 56, 77)81.5285(32, 25, 51, 74, 36, 35, 22, 76)101.3951(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)RMSE (o)21.9255(7, 22)41.9554(23, 36, 34, 41)61.9213(9, 61, 31, 23, 41, 70)81.8391(17, 55, 47, 54, 66, 39, 41, 28)101.8712(15, 54, 77, 37, 15, 78, 12, 19, 65, 46)Bold values represent the best model for each performance indicator.It can be seen that the DNN model with 10 hidden layers has evolved to bring about better performance; the best generation yielded a correlation ofR2 = 0.938, MAE = 1.395, and RMSE = 1.846. The model with the next best performance is the DNN model with 8 hidden layers which produce accurate intermediate precision (R2 = 0.931, MAE = 1.528, and RMSE = 1.839). The DNN model with 10 hidden layer gives the best performance for R2 and MAE criteria, while the model with 8 hidden layers gives the best result for the RMSE cost function. However, the best neuron structure for each cost function is different and is also shown in Table 6. The evolutions of number of neurons in 10 hidden layers and 8 hidden layers of the best DNN model referring to R2, MAE, and RMSE are presented in Figure 10. It means that the evolution of the model has found three good results; choosing the best model will be done in the next section.Figure 10
Evolution of neurons in each hidden layer using PSO-DNN models: (a) 10 hidden layers,R2; (b) 10 hidden layers, MAE; (c) 8 hidden layers, RMSE (HD∗: hidden layers).
(a)(b)(c)


## 4.3. Predictive Capability of the Models

In this section, the three best models, 10 hidden layers,R2 (10-HD-R2), 10 hidden layers, MAE (10-HD-MAE), and 8 hidden layers, RMSE (8-HD-RMSE) were chosen for comparison with each other. From the point of view of statistical probability, the random factors in the input variables should be considered. Therefore, 300 simulations take into account the randomness of the input variable sequence number. The data used in this section include training, validation, and testing data sets. The criteria for selecting the best model are based on the results of the testing data set. The results are shown in Figure 11 and Tables 7–9 .Figure 11
Performance of PSO-DNN models using criteria:R2 (a), MAE (b), and RMSE (c).
(a)(b)(c)Table 7
Summary of the 300 simulations usingR2 criteria.
ModelData setAverageMinMaxSD10-HD-R2Training0.9460.9150.9700.0083Validation0.8950.7210.9390.0377Testing0.9350.8610.9660.013810-HD-MAETraining0.9000.5860.9530.0477Validation0.8510.4360.9300.0657Testing0.8890.5490.9390.04538-HD-RMSETraining0.9350.8260.950.0156Validation0.9010.6220.9420.0343Testing0.8810.7950.9270.0186Table 8
Summary of the 300 simulations using MAE criteria.
ModelData setAverage (o)Min (o)Max (o)SD10-HD-R2Training1.1880.8831.4230.0899Validation1.7211.3922.5090.2345Testing1.3401.0361.7150.133910-HD-MAETraining1.6361.0502.5600.3031Validation2.1051.4733.4550.3411Testing1.8231.2432.8700.2718-HD-RMSETraining1.2681.1012.1070.1534Validation1.7061.2673.4440.2564Testing1.5231.2382.3380.1923Table 9
Summary of the 300 simulations using RMSE criteria.
ModelData setAverage (o)Min (o)Max (o)SD10-HD-R2Training1.5661.2601.8480.1088Validation2.2741.7823.4980.3395Testing1.771.4062.5920.183510-HD-MAETraining2.0591.4213.1280.3711Validation2.6681.8765.0950.473Testing2.2551.5783.3100.29888-HD-RMSETraining1.7221.4653.5980.2129Validation2.221.6764.8670.3861Testing2.0191.6133.8820.3264The results showed that the predicted results on the training and validation sets gave good accuracy with satisfactory average accuracyR2 ranging from 0.851 to 0.946, MAE ranging from 1.188 to 2.105, and RMSE varying from 1.566 to 2.668, in which it can be seen that, among three good models, the 10-HD-R2 gives the best prediction, which yielded an average correlation of R2 = 0.935, MAE = 1.34, and RMSE = 1.77 on the testing data set. The next best model is 8-HD-RMSE, which produces accurate intermediate precision (R2 = 0.881, MAE = 1.523, and RMSE = 2.019). Moreover, the standard deviation of the 10-HD-R2 model is better than that of the 8-HD-RMSE, where SD = 0.0138, 0.1339, and 0.1835 for R2, MAE, and RMSE criteria compared to SD = 0.0186, 0.1923, and 0.3264 of two models, respectively.Table10 shows the comparison between the DNN model and the PSO-DNN model. The DNN model received initialized parameters as shown in Table 11. The best 10-HD-R2 PSO-DNN model was included for comparison with the proof of the effectiveness of the algorithm. The average error criteria of 300 simulations were compared to confirm the model’s performance. It can be seen that the 10-HD-R2 PSO-DNN model outperformed the DNN model, which improved the average correlation of R2 by 1.83%, MAE by 5.94%, and RMSE by 8.58%.Table 10
Comparison between the DNN model and the PSO-DNN model.
CriteriaDNN modelPSO-DNN modelPerformance improvement (%)R20.9180.9351.83MAE (o)1.4251.3405.94RMSE (o)1.9361.7708.58Table 11
Comparison with previous studies.
AuthorModelParameterNumber of samplesRR2RMSE (o)MAE (o)Khanlari et al. [60]ANNInternal friction angle2000.890.7922.261.92MVR0.890.792.3851.929Mohammadi et al. [61]MLPInternal friction angle1080.9020.8149.2856.137MLR0.7190.5175.0823.954Iyeke et al. [62]ANNInternal friction angle830.8970.8054.774.34Present studyDNNInternal friction angle2450.9580.9181.9361.425PSO-DNN0.9670.9351.771.34Table11 presents some research results on ML applications in determining the internal friction angle of soil. The results of the present study, as well as other studies, show the effectiveness of the ML technique in predicting the internal friction angle of the soil with expected R2 from 0.79 to 0.935 on the testing data set. However, due to the different data sets used and the input variables, it makes no sense to compare these results. A project using different data sets as well as various input variables is needed to provide a general model for the prediction of soil shear parameters.Figure12 shows a visual comparison of test results and predictions based on the friction angle of soil of the DNN and PSO-DNN models. The performances of two ML models have been tested on all three data sets: training, validation, and testing.Figure 12
Measured and predicted values of friction angle of soil using the DNN models: PSO-DNN model for (a) training, (c) validation, and (e) testing data set; DNN model for (b) training, (d) validation, and (f) testing data set.
(a)(b)(c)(d)(e)(f)


## 4.4. Sensitivity Analysis

In this section, a global sensitivity analysis was conducted to evaluate the importance of input parameters for the model using Monte Carlo methods [63]. It is an effective way to investigate a relationship between input and output. The input data set taken from Saltelli’s sampling scheme was used to develop the DNN model [64]. The global sensitivity index is determined by the following formulas:(4)VarY=∑i=1dVari+∑i<jdVarij+∑j<mdVarijm+…+Varij…d,(5)STi=Vari+∑i<jdVarij+∑j<mdVarijm+…+Varij…dVarY,where Var(Y) denotes the total variance of the model output; d denotes the number of input features; Vari denotes the model output variance in response to variation of the ith input variable; Varij denotes the model output variance in response to the simultaneous variation of the ith and the jth input; STi denotes the total sensitivity index.The result of the total sensitivity analysis was shown in Figure13. It can be seen that, among the 7 input variables used to predict the friction angle of soil, the elevation of the soil layer bottom (Z2) was the most important feature, which achieved an average sensitivity index score of 0.625. The topsoil layer elevation (Z1) was the second important variable, confirmed by an average sensitivity index score of 0.603. From the point of view of soil mechanics, the deeper the soil is, the more compact it is, so the depth of the soil sample plays an important role in detecting the friction angle of the soil when its density, state, and many other mechanical properties change according to the depth. The variables H, G, and S were ranked as the third to the fifth important predictors, with an average sensitivity index ranging from 0.203 to 0.05. Other predictor variables included in the model (e, G) had lower than 0.01 in the sensitivity index, indicating that they had not affected the output prediction. It is important to note that the sensitivity analysis in this section is only relevant for the input data set itself and cannot be confirmed with other data sets.Figure 13
Sensitivity analysis using the 10-HD-R2 model.


## 5. Conclusions

In this study, a PSO-DNN hybrid model, which can evolve itself to find out the best models, was developed to predict the friction angle of soil. The hybrid model evolved on its own and found three of the best among the survey models. A database containing 245 soil samples from geological boreholes was used to develop and evaluate the three proposed DNN models: 10-HD-R2, 10-HD-MAE, and 8-HD-RMSE.The results show that the models’ performances improved and stabilized fromR2 = 0.602 to 0.889 on the validation set with the increase in training data set size from 20% to 60%. Research results show that PSO-DLNN obtains the best results with the number of hidden layers from 8 to 10. The optimal number of neurons in each hidden layer is not the same and is distributed intricately in the hidden layer. It recommends that a DLNN model with 8 to 10 hidden layers might be optimal for the problem related to predicting the friction angle of soil. However, it is advisable to select the number of neurons in each hidden layer through evolutionary methods to bring about high efficiency to the DLNN model. The results also showed that, on the training and validation data sets, all three best models, 10-HD-R2, 10-HD-MAE, and 8-HD-RMSE, have good predict results, in which the leading is the 10-HD-R2 model on the testing data set; the 10-HD-R2-DNN model still gave the best results and outperformed the other two models. Prediction results of 300 simulations show that the 10-HD-R2-DNN model has a smaller standard deviation, indicating that the model is more stable than the other two models.In addition, the sensitivity analysis using the Monte Carlo method was carried out to evaluate the importance of input features in the model study. The results show that the two inputs related to the depth of the soil layers (Z2 and Z1) were considered to be the most important parameters for predicting soil friction angle.



---

*Source: 5570945-2021-05-07.xml*