# Compressive Strength Prediction Using Coupled Deep Learning Model with Extreme Gradient Boosting Algorithm: Environmentally Friendly Concrete Incorporating Recycled Aggregate

Authors: Mayadah W. Falah; Sadaam Hadee Hussein; Mohammed Ayad Saad; Zainab Hasan Ali; Tan Huy Tran; Rania M. Ghoniem; Ahmed A. Ewees
Journal: Complexity
 (2022)

---

## Abstract

The application of recycled aggregate as a sustainable material in construction projects is considered a promising approach to decrease the carbon footprint of concrete structures. Prediction of compressive strength (CS) of environmentally friendly (EF) concrete containing recycled aggregate is important for understanding sustainable structures’ concrete behaviour. In this research, the capability of the deep learning neural network (DLNN) approach is examined on the simulation of CS of EF concrete. The developed approach is compared to the well-known artificial intelligence (AI) approaches named multivariate adaptive regression spline (MARS), extreme learning machines (ELMs), and random forests (RFs). The dataset was divided into three scenarios 70%-30%, 80%-20%, and 90%-10% for training/testing to explore the impact of data division percentage on the capacity of the developed AI model. Extreme gradient boosting (XGBoost) was integrated with the developed AI models to select the influencing variables on the CS prediction. Several statistical measures and graphical methods were generated to evaluate the efficiency of the presented models. In this regard, the results confirmed that the DLNN model attained the highest value of prediction performance with minimal root mean squared error (RMSE = 2.23). The study revealed that the highest prediction performance could be attained by increasing the number of variables in the prediction problem and using 90%-10% data division. The results demonstrated the robustness of the DLNN model over the other AI models in handling the complex behaviour of concrete. Due to the high accuracy of the DLNN model, the developed method can be used as a practical approach for future use of CS prediction of EF concrete.


---

## Body

## 1. Introduction

Consideration of sustainable development is an important requirement in the new era of the construction industry [1, 2]. Sustainable development is applied by reducing environmental impact and protecting natural resources. Increasing population growth led to the high demand for construction and, as a result, depletion of natural resources needed in concrete construction [3, 4]. Recently, many researchers have studied the production of environmental friendly (EF) concrete, which has less harmful effects on nature [5–8]. One of the methods studied by the researchers is using recycled aggregate [9]. Production of construction debris and waste has been increasing using recycled materials in concrete production in the last years. The use of recycled aggregate in a construction project is considered a promising method to decrease the carbon footprint of a concrete project [10–13]. In recent years, EF concrete has been studied experimentally and mathematically [14, 15].Early investigation on the mechanical behaviour of EF concrete was conducted using stone dust in concrete production [16]. The modelling results showed that the compressive strength of EF concrete is higher. The durability of concrete was studied and confirmed to produce high performance using recycled material [14]. Two researches have adopted an investigation of experimental analysis for the effect of using additive material such as fly ash and silica fume beside recycled aggregate on the mechanical properties of concrete [17, 18]. The studies indicated that using additive materials and recycled aggregate can enhance the strength and durability of concrete. Recycled aggregate usage in concrete enhances pore distribution and frost resistance [19]. The addition of agriculture waste to the concrete mix was considered and was investigated to enhance the properties of concrete using silica fume [20]. Calcium carbide residue was examined over the literature as industrial material [21]. Compressive strength and failure modes of concrete containing recycled nylon fiber fabric were studied [22]. Other researchers confirmed the potential of the produced EF concrete, and some researchers studied the utilization of agriculture and local waste in Africa for concrete production [23]. Others discussed the properties of lightweight interlocking concrete by adding sawdust and laterite in the concrete production [24]. All the reported literature on the experimental developed EF concrete elaborated the significance of producing such product material that can help essentially on the construction field and material sustainability. In addition, studies showed the importance of using recycled waste as environmental material in concrete production.Owing to the complexity and nonlinearity of concrete behaviour and various parameters that affect CS, the necessity of developing an effective technique that can examine the properties of concrete can be noted. In recent years, artificial intelligence (AI) models have been effectively applied in concrete structure research [25–29]. AI models have the ability to solve nonlinear, stochastic problems and deal with complex systems [30–32]. Literature studies on AI models reported massively in this domain; artificial neural network (ANN) model with response surface methodology (RSM) was developed for predicting CS of recycled aggregate concrete [33]. The study concluded that the ANN-based model exhibits better performance than the RSM model. The development of adaptive neuro-fuzzy inference system, ANFIS, together with ANN model was adopted to estimate self-compacting concrete’s CS [34]. The results demonstrated that the developed model accurately predicted concrete compressive strength. In another study, adaptive boosting (AdaBoost), ANN and support vector machine (SVM) were developed for the prediction process of compressive strength [35]. The study demonstrated that AdaBoost model exhibited better prediction achievement than other models. Examination of the feasibility of ANN model in the prediction process of concrete CS involving furnace slag of ground granulated blast was conducted by [36]. The study was integrated ANN with a multiobjective salp optimization algorithm and compared it with M5P model for CS prediction. The research was concluded that the proposed AI models successfully predicted CS. The incorporation of socio-political algorithm (ICA) with extreme gradient boosting (XGBoost), ANN, SVM, and ANFIS was tested for predicting of CS of recycled aggregate concrete, and was reported by [26]. The study revealed that the developed model (ICA-XGBoost) achieved better capacity than the other proposed models in the prediction process. The ability of ANN model was evaluated in the prediction of CS of Geopolymer Concrete (GPC) [37]. The results indicated that the proposed model attained high prediction capability of compressive strength. Boosted decision tree regression (BDTR) and SVM models were developed to predict EF concrete’s compressive strength [38]. The researcher concluded that the BDTR model performed better than the SVM model with good accuracy prediction. The modelling accuracy of ANN, RSM, and gene expression programming (GEP) was tested for predicting the CS of engineering Geopolymer composites concrete [39]. The study showed that ANN and RSM models performed better than GEP model in the prediction process. Hybridization of three optimization algorithms called genetic algorithm (GA), salp swarm optimization (SSA), and grasshopper optimization algorithm (GOA) with ANN model were conducted for predicting CS of the concrete utilizing recycled aggregate [40]. The researchers indicated that ANN-SSA achieved better prediction performance than other models. Several other researches were adopted on testing the feasibility of the AI models for modelling the CS of concrete and confirmed the potential of those new computer-aided models in solving such a kind of complex material engineering problem [41–45].Previous studies show that the use of recycled aggregate as a sustainable material in concrete production is highly emphasized by several researchers [46, 47]. Most of these studies examined the impact of recycled aggregate experimentally and there is a limited number of researchers who have explored the properties of EF concrete by a mathematical model. Due to the rapid advancement of AI models and computer vision, exploring a new approach that can handle the complex relationship between different parameters and the compressive strength of concrete is very important. Deep learning neural network (DLNN) is a new version of AI models that have achieved a reliable performance in solving complex and nonlinear systems [48–50]. DLNN outperformed other traditional AI models in various applications owing to their capacity on analyzing through the deep layers learning process. DLNN has been successfully applied and performed better prediction precision in several engineering problems [51–54]. The other concern that highly matters to the machine learning modelling is the feature selection for the appropriate predictors. Hence, the current research focused on integrating a new feature selection approach.This paper aims to provide an effective computer-aided model for predicting CS of recycled aggregate concrete. This approach can help determine the performance of EF concrete on the improvement of concrete strength and identify the appropriate mix designs of concrete. To achieve the research aim, the DLNN model is developed and compared with extreme learning machine (ELM), multivariate adaptive regression spline (MARS), and random forest (RF). Due to the important role of input parameters in CS prediction, XGBoost as an advanced algorithm is used that abstracts the most correlated variables in the modelling process. The proposed approach is helpful in many applications of a concrete structure and can be applied to examine the compressive strength of EF concrete.


## 2. Methodology Overview

### 2.1. Deep Learning Neural Network (DLNN)

In recent years, the concept of the deep learning method has been introduced as an advanced algorithm of neural networks. Deep neural network is a traditional neural network with extra numbers of hidden layers that are added to the structure [55]. The deep learning algorithm was developed by Hinton et al. as he introduced a layer-wise-greedy-learning method [56]. According to this method, an unsupervised learning method is used to pertain the neural network before the training phase with layer by layer. Herein, deep learning is a popular AI model for some reasons: its ability to deal with big training data, avoid overfitting problems, and non-random value can be assigned to the network before learning process [55]. From these features, the algorithm can make a good performance through the training process among deep learning types. The backpropagation neural network is used in this study due to its popularity and robustness in many applications [51]. This type used multiple hidden layers and backpropagation with gradient descent algorithm. The main concept of this algorithm is yielding new variables based on the connection between hidden layers and the input layers. The new variables are reached to the output layer, and then the target value is predicted through the training process. The nonlinear relationship between multiple hidden layers gave the algorithm the capability to handle nonlinear relationships in different complex systems [57]. The structure of a deep neural network is shown in Figure 1 with the input layer, hidden layers, and output layer. The mathematical expression of the algorithm is discussed as follows:(1)MI=fwIφI−1+bI,for0<I<L,where f is the activation function, b is the bias, and w is the weight matrix of the hidden neurons. Input layer is described by 0 and L represents the output layer. The activation matrix used in this study is tanh function because of its capability to get good prediction performance in the explored problem.Figure 1
The architecture of the constructed DLNN model.


### 2.2. Multivariate Adaptive Regression Spline Model (MARS)

Multivariate adaptive regression spline model, a robustn machine learning model, was developed by [58]. MARS is a nonlinear flexible model recently used in many applications in engineering subjects [59]. The MARS model has three principle components: the basic functions (BFs), spline functions, and the knots [60]. The relationship between target value and predicted variables is addressed by BFs and it is expressed by max 0,c−x or max 0,x−c, where x represents threshold value and c indicates the value of input variable. In this context, the knots indicate the function for the base as well as base endpoints. Spline functions contain one or more BFs and its role is developing a regression model for each node [61]. The predicted value in MARS model is based on linear combination of BFs components. The main processes of MARS model are as follows: Assuming Y represents the target variable whereas X is the matrix of input variable, then the mathematical expression of MARS model is described as follows:(2)Y=fX=β0+∑m=1Mβ0BFmX,where β0 represents the initial value; and BFm is the basis function that is used to fit MARS model; whereas M denotes the whole number of BFs [62]. There are two main phases in MARS model named forward phase and backward phase as shown in Figure 2. In forward phase, the model selects the optimum combination of input parameters. The overfitting problem can appear in the forward phase due to a series of splits, and the model cannot achieve good performance during the prediction process. The model uses the backward phase to remove undesirable parameters that have been selected and then enhance the prediction accuracy of a regression model. The basis deletion criteria in the backward phase are a generalized cross validation, which is indicated as GCV and can be computed as follows:(3)GCVM=1/N∑i=1NOi−fxi21−CM/N2,CM=d+1×M,where Oi denotes the observed value; N is the total number of data; fxi is the predicted value of x, M represents the whole number of BFs; and CM indicates the penalty factor. d represents the optimization cost of BFs and its range 2≤d≤4. The systematic structure of MARS model is described in Figure 2.Figure 2
MARS model architecture.


### 2.3. Extreme Learning Machine (ELM)

ELM is one of the advanced models that have been used in recent years to train the developed single layer feedforward neural network [63]. Each layer in ELM model has a number of neurons and the model contains input, hidden, and output layer. The sigmoid function is commonly utilized as activation function for the hidden layer, while the activation function in input and output layer is linear function [64]. To achieve the optimum linear system solution, developing the ELM model required the determination of random input weight and hidden biases, then using the Moore Penrose generalized inverse method [65, 66]. ELM model has been used in many engineering tasks owing to fast learning, no parameter tuning, and its strong ability to achieve generalized model [67–69]. In ELM model, the first step is linear mapping the input vectors into an L-dimensional feature map as shown below:(4)t˜i=∑i=1Lβi.gwl.xi+bl,i=1,2,…,N,where N denotes the size of training data, t˜i represents the output vector, and xi the input vector. βi refers to the weight vector which represents the connection between hidden neuron and output layer; wl is the weight vector that links the hidden neuron to the first layer; bl represents the bias; while g symbolizes the activation function. In this regard, the structure of ELM model is illustrated in Figure 3.Figure 3
Paradigm of extreme learning machine model.According to the ELM model, a traditional single layer ANN can handle all sample data zero variation as shown in the below mathematical expression:(5)∑i=1Nti−t˜i=∑i=1Nti−∑l=1Lβi.gwl.xi+bl=0,where ti represents the output vector. xi denotes the input vector. Herein, the previous equation can be reconstructed as shown below:(6)Hβ=T,where(7)H=gw1⋅x1+b1⋯gwL⋅x1+bL⋮⋱⋮gw1⋅xN+b1⋯gwL⋅xN+bLN×I,β=β1,1⋯β1,m⋮⋱⋮βL,1⋯βL,mL×m,T=t1,1⋯t1,m⋮⋱⋮tN,1⋯tN,mN×m,where β indicates the weight for the matrix which links hidden and output layer; H is output matrix of hidden layer; and T is the output matrix of target value based on N training data.By assuming that hidden biases along with input weight are both constant; it suggests that the model can be described as linear system whereH and T indicate an output matrix from hidden layer and output target. While β is the weight of the matrix optimized during the training phase. The least-squares solution of the linear system can be expressed as follows:(8)β˜=H†T,where H† represents Moore Penrose generalized inverse matrix.


### 2.4. Random Forest (RF)

Random forest (RF) is a robust AI method based on classification and regression tree and developed by [70]. RF model was applied in classification and regression problems and gave an excellent performance in many fields of engineering applications [71, 72]. The work of RF model is based on the abstraction of multiple samples from the original sample by using the bootstrap resampling method, and then a decision tree model is developed for each bootstrap sample, and various outputs of decision tree models are combined to get the final result of prediction. Modifying the model parameters led to the use of many types of decision trees and then developed different trees models.The modelling process of RF algorithm includes the following phases: using bootstrap resampling method to extractN predictors training data from the original problem data [73]. During training phase, the training data of RF model should be equal to 2/3 of the original data size. The utilized data is called in-bag data and the remaining 1/3 dataset is called out-of-bag data. The regression tree (RT) is developed for each training bootstrap data and a random forest algorithm is built based on the developing N predictors of RT models [74]. The variation of regression trees can be determined during the training data process by randomly selecting the optimal attribute from the maximum depth attributes; this process increases the capability of the RF model. The sequence of regression model can be attained by training the algorithm several times and this sequence is helpful for the developing of random forest system. The prediction results of N predictors are collected and final regression model for the new sample is calculated by using the average method [75]. The mathematical equation of regressions model in RF algorithm is as follows:(9)f^rfKx=1K∑k=1Ktix,where f^rfKx represents the regression model of random forest, ti is the regression model of each regression tree, and K represents the number of regression tree model. The structure of RF model is shown in Figure 4:Figure 4
Random Forest model structure.


### 2.5. Extreme Gradient Boosting (XGBoost)

XGBoost is an advanced algorithm that was used recently in the feature selection process. It was developed as an extension and improvement of gradient boosting tree [76, 77]. The main structure of the algorithm depends on the efficient construction of boosted trees [78]. In feature selection problem, the main objective of XGBoost is to construct boosted trees to achieve the feature importance of input variables that are used for training process [79]. Boost trees are classified into classification and regression trees. The features’ importance is extracted using three methods: gain, frequency, and cover [80]. The gain method calculates the importance of the features, frequency computes the number of trees in the boosted trees, and cover calculates the relative value for the observation [79]. The feature importance is calculated by weight using XGBoost as follows [81]:(10)Nv=∑L=1L∑l=1X=1IVLl,v,VLl,v=fx=1ifVLl=v0,otherwise,where L represents the number of trees, N indicates the number of leaf nodes, VLl is the feature of node l, and I is the indicator function.


## 2.1. Deep Learning Neural Network (DLNN)

In recent years, the concept of the deep learning method has been introduced as an advanced algorithm of neural networks. Deep neural network is a traditional neural network with extra numbers of hidden layers that are added to the structure [55]. The deep learning algorithm was developed by Hinton et al. as he introduced a layer-wise-greedy-learning method [56]. According to this method, an unsupervised learning method is used to pertain the neural network before the training phase with layer by layer. Herein, deep learning is a popular AI model for some reasons: its ability to deal with big training data, avoid overfitting problems, and non-random value can be assigned to the network before learning process [55]. From these features, the algorithm can make a good performance through the training process among deep learning types. The backpropagation neural network is used in this study due to its popularity and robustness in many applications [51]. This type used multiple hidden layers and backpropagation with gradient descent algorithm. The main concept of this algorithm is yielding new variables based on the connection between hidden layers and the input layers. The new variables are reached to the output layer, and then the target value is predicted through the training process. The nonlinear relationship between multiple hidden layers gave the algorithm the capability to handle nonlinear relationships in different complex systems [57]. The structure of a deep neural network is shown in Figure 1 with the input layer, hidden layers, and output layer. The mathematical expression of the algorithm is discussed as follows:(1)MI=fwIφI−1+bI,for0<I<L,where f is the activation function, b is the bias, and w is the weight matrix of the hidden neurons. Input layer is described by 0 and L represents the output layer. The activation matrix used in this study is tanh function because of its capability to get good prediction performance in the explored problem.Figure 1
The architecture of the constructed DLNN model.


## 2.2. Multivariate Adaptive Regression Spline Model (MARS)

Multivariate adaptive regression spline model, a robustn machine learning model, was developed by [58]. MARS is a nonlinear flexible model recently used in many applications in engineering subjects [59]. The MARS model has three principle components: the basic functions (BFs), spline functions, and the knots [60]. The relationship between target value and predicted variables is addressed by BFs and it is expressed by max 0,c−x or max 0,x−c, where x represents threshold value and c indicates the value of input variable. In this context, the knots indicate the function for the base as well as base endpoints. Spline functions contain one or more BFs and its role is developing a regression model for each node [61]. The predicted value in MARS model is based on linear combination of BFs components. The main processes of MARS model are as follows: Assuming Y represents the target variable whereas X is the matrix of input variable, then the mathematical expression of MARS model is described as follows:(2)Y=fX=β0+∑m=1Mβ0BFmX,where β0 represents the initial value; and BFm is the basis function that is used to fit MARS model; whereas M denotes the whole number of BFs [62]. There are two main phases in MARS model named forward phase and backward phase as shown in Figure 2. In forward phase, the model selects the optimum combination of input parameters. The overfitting problem can appear in the forward phase due to a series of splits, and the model cannot achieve good performance during the prediction process. The model uses the backward phase to remove undesirable parameters that have been selected and then enhance the prediction accuracy of a regression model. The basis deletion criteria in the backward phase are a generalized cross validation, which is indicated as GCV and can be computed as follows:(3)GCVM=1/N∑i=1NOi−fxi21−CM/N2,CM=d+1×M,where Oi denotes the observed value; N is the total number of data; fxi is the predicted value of x, M represents the whole number of BFs; and CM indicates the penalty factor. d represents the optimization cost of BFs and its range 2≤d≤4. The systematic structure of MARS model is described in Figure 2.Figure 2
MARS model architecture.


## 2.3. Extreme Learning Machine (ELM)

ELM is one of the advanced models that have been used in recent years to train the developed single layer feedforward neural network [63]. Each layer in ELM model has a number of neurons and the model contains input, hidden, and output layer. The sigmoid function is commonly utilized as activation function for the hidden layer, while the activation function in input and output layer is linear function [64]. To achieve the optimum linear system solution, developing the ELM model required the determination of random input weight and hidden biases, then using the Moore Penrose generalized inverse method [65, 66]. ELM model has been used in many engineering tasks owing to fast learning, no parameter tuning, and its strong ability to achieve generalized model [67–69]. In ELM model, the first step is linear mapping the input vectors into an L-dimensional feature map as shown below:(4)t˜i=∑i=1Lβi.gwl.xi+bl,i=1,2,…,N,where N denotes the size of training data, t˜i represents the output vector, and xi the input vector. βi refers to the weight vector which represents the connection between hidden neuron and output layer; wl is the weight vector that links the hidden neuron to the first layer; bl represents the bias; while g symbolizes the activation function. In this regard, the structure of ELM model is illustrated in Figure 3.Figure 3
Paradigm of extreme learning machine model.According to the ELM model, a traditional single layer ANN can handle all sample data zero variation as shown in the below mathematical expression:(5)∑i=1Nti−t˜i=∑i=1Nti−∑l=1Lβi.gwl.xi+bl=0,where ti represents the output vector. xi denotes the input vector. Herein, the previous equation can be reconstructed as shown below:(6)Hβ=T,where(7)H=gw1⋅x1+b1⋯gwL⋅x1+bL⋮⋱⋮gw1⋅xN+b1⋯gwL⋅xN+bLN×I,β=β1,1⋯β1,m⋮⋱⋮βL,1⋯βL,mL×m,T=t1,1⋯t1,m⋮⋱⋮tN,1⋯tN,mN×m,where β indicates the weight for the matrix which links hidden and output layer; H is output matrix of hidden layer; and T is the output matrix of target value based on N training data.By assuming that hidden biases along with input weight are both constant; it suggests that the model can be described as linear system whereH and T indicate an output matrix from hidden layer and output target. While β is the weight of the matrix optimized during the training phase. The least-squares solution of the linear system can be expressed as follows:(8)β˜=H†T,where H† represents Moore Penrose generalized inverse matrix.


## 2.4. Random Forest (RF)

Random forest (RF) is a robust AI method based on classification and regression tree and developed by [70]. RF model was applied in classification and regression problems and gave an excellent performance in many fields of engineering applications [71, 72]. The work of RF model is based on the abstraction of multiple samples from the original sample by using the bootstrap resampling method, and then a decision tree model is developed for each bootstrap sample, and various outputs of decision tree models are combined to get the final result of prediction. Modifying the model parameters led to the use of many types of decision trees and then developed different trees models.The modelling process of RF algorithm includes the following phases: using bootstrap resampling method to extractN predictors training data from the original problem data [73]. During training phase, the training data of RF model should be equal to 2/3 of the original data size. The utilized data is called in-bag data and the remaining 1/3 dataset is called out-of-bag data. The regression tree (RT) is developed for each training bootstrap data and a random forest algorithm is built based on the developing N predictors of RT models [74]. The variation of regression trees can be determined during the training data process by randomly selecting the optimal attribute from the maximum depth attributes; this process increases the capability of the RF model. The sequence of regression model can be attained by training the algorithm several times and this sequence is helpful for the developing of random forest system. The prediction results of N predictors are collected and final regression model for the new sample is calculated by using the average method [75]. The mathematical equation of regressions model in RF algorithm is as follows:(9)f^rfKx=1K∑k=1Ktix,where f^rfKx represents the regression model of random forest, ti is the regression model of each regression tree, and K represents the number of regression tree model. The structure of RF model is shown in Figure 4:Figure 4
Random Forest model structure.


## 2.5. Extreme Gradient Boosting (XGBoost)

XGBoost is an advanced algorithm that was used recently in the feature selection process. It was developed as an extension and improvement of gradient boosting tree [76, 77]. The main structure of the algorithm depends on the efficient construction of boosted trees [78]. In feature selection problem, the main objective of XGBoost is to construct boosted trees to achieve the feature importance of input variables that are used for training process [79]. Boost trees are classified into classification and regression trees. The features’ importance is extracted using three methods: gain, frequency, and cover [80]. The gain method calculates the importance of the features, frequency computes the number of trees in the boosted trees, and cover calculates the relative value for the observation [79]. The feature importance is calculated by weight using XGBoost as follows [81]:(10)Nv=∑L=1L∑l=1X=1IVLl,v,VLl,v=fx=1ifVLl=v0,otherwise,where L represents the number of trees, N indicates the number of leaf nodes, VLl is the feature of node l, and I is the indicator function.


## 3. Data Collection and Model Development

For the development of the proposed AI model, datasets were taken from open-source datasets [82]. The datasets contain 353 parameters of environmentally friendly concrete samples and their compressive strength. The datasets include water, (C: cement), (FA: fine aggregate), (CA: coarse aggregate), (RA: recycled aggregate), (AS: age strength), and (CS: compressive strength). The statistical characteristics of the datasets are reported in Table 1. These statistical characteristics include maximum value, minimum value, mean, standard deviation, skewness, and kurtosis. Figure 5 demonstrates the histogram for all input and output variables. The CS of the EF concrete ranges from 13 to 88.3, with mean value equal to 42.11, indicating high variance and giving the noticeable complexity of prediction problem. Table 1 and Figure 5 show that the data are well distributed and nearly to normal distribution.Table 1
Statistical measures of the studied dataset.
ParametersMinimumMaximumMeanStd. deviationSkewnessKurtosisWater (kg/m3)120244184.127.1360.2850.3497C (kg/m3)220750394.983.9421.1012.889FA (kg/m3)3651020710.3108.4520.3131.435CA (kg/m3)01366564.8438.708-0.099-1.458RA (kg/m3)01259504.3414.5880.218-1.418AS (days)718045.2543.9561.7682.174CS (MPa)1388.342.1113.1260.6390.728Figure 5
Histograms of the dataset used in the study.The correlation matrix between input and output variables is depicted in Figure6. Correlation statistics showed a good correlation between cement quantity and compressive strength. Increased cement quantity led to increasing the compressive strength of concrete. Figure 6 also demonstrates that there is a small correlation between value of coarse aggregate and compressive strength. There is also a small negative correlation between water, fine aggregate, recycled aggregate, and compressive strength. Increases of water, fine aggregate, and recycled aggregate decreases the compressive strength of concrete.Figure 6
Correlation statistics between concrete parameters in the dataset.Four AI models were adopted for the modelling process, i.e., DLNN, ELM, MARS, and RF. Dataset was split into three scenarios including 70-30%, 80-20%, and 90-10% to examine the best size of training and testing dataset suitable for modelling problems. XGBoost was used as a robust model to abstract the highly correlated variables of the best size of training and testing dataset for CS prediction. The input combinations of CS prediction are reported as in Table2. It can be noted that the most relevant variable of CS prediction is the quantity of cement. The second input combinations of CS prediction are quantity of cement and water. Input combinations of XGBoost model showed that negatively correlated parameters and cement quantity are important variables in CS prediction. The procedure of the developed AI model is illustrated in Figure 7.Table 2
The abstracted input combination using XGBoost model.
ModelsInput combinationsModel ICS=CModel IICS=C,WaterModel IIICS=C,Water,FAModel IVCS=C,Water,FA,CAModel VCS=C,Water,FA,CA,ASModel VICS=C,Water,FA,CA,AS,RAFigure 7
The developed AI models for CS prediction EF concrete.The accuracy of modelling is evaluated by using different statistical indicators including determination coefficient (R2), mean absolute percentage error (MAPE), root mean square error (RMSE), mean absolute error (MAE), Nash–Sutcliffe efficiency (NSE), and Willmott’ index (WI) [83, 84].(11)R2=∑i=1Nyp−yp¯⋅yo−yo¯∑i=1Nyp−yp¯2∑i=1Nyo−yo¯22,MAPE=1n∑i=1nyp−yoyo,RMSE=∑i=1Nyp−yo2n,MAE=∑i=1Nyp−yoN,NSE=1−∑i=1Nyp−yo2∑i=1Nyo−yo¯2,WI=1−∑i=1Nyp−yo2∑i=1Nyp−yo¯+yo−yo¯2,where yp and yo represent the predicted and observed value of compressive strength; yo¯ is the mean value of the observed value of compressive strength; and N represents the number of experiment samples.


## 4. Results and Discussion

In this research, four AI models were established to predict the compressive strength of EF concrete containing recycled aggregate. The main idea from developing AI models is to examine the ability of the developed model to predict the compressive strength of EF concrete, and therefore three data division scenarios were used to test the size of dataset suitable for CS prediction.Tables2–4 describe the statistical indicators of the developed models (DLNN, MARS, ELM, and RF) over the training phase for the three data division scenarios. In general, the results indicated that all the developed models performed good prediction performance. This indicated that the developed models are reliable and they are capable to build a robust approach of CS prediction. However, the reported results showed the superiority of DLNN model over the other developed models through all data division scenarios. The prediction performance is gained by using DLNN model for 80-20% data division scenario with R2 = 0.99, MAPE = 0.02, RMSE = 1.29, MAE = 0.74, NSE = 0.98, and WI = 0.99 (Tables 3 and 4).Table 3
The statistical measurements for the developed AI model over the training phase for 70-30% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.980.031.741.070.981.00MARS0.880.094.603.570.880.97ELM0.930.073.622.610.930.98RF0.870.104.983.790.860.96Table 4
The statistical measurements for the developed model AI model over the training phase for 80-20% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.990.021.300.740.991.00MARS0.860.104.853.810.860.96ELM0.910.073.982.870.910.98RF0.880.094.703.590.870.96The results of test phase are shown in Tables5–7 with respect to the developed AI models in all scenarios. The results reported that the DLNN model performed the best predictive accuracy over the three adopted scenarios. The results indicated that using different data division scenarios has an essential role in training the developed AI model. Using 90% of dataset for the training process contributes to improving the predictive performance of AI model. The best prediction accuracy is attained by DLNN model and 90-10% scenario with R2 = 0.97, MAPE = 0.04, RMSE = 2.23, MAE = 1.63, NSE = 0.93, and WI = 0.99 (Tables 6–8).Table 5
Statistical measures for the developed AI model across the training phase for 90-10% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.980.042.011.370.980.99MARS0.880.094.593.450.880.97ELM0.910.084.012.890.910.98RF0.890.094.543.410.880.96Table 6
Statistical measurements for the developed AI model in the test phase by using 70-30% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.900.084.002.990.900.97MARS0.800.136.044.750.760.94ELM0.650.168.215.630.560.89RF0.890.084.333.190.880.96Table 7
Statistical measurements for the developed AI model in the test phase by using 80-20% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.940.063.372.350.940.98MARS0.830.105.584.140.830.95ELM0.820.125.814.350.810.95RF0.890.084.673.320.880.96Table 8
Statistical measurements for the developed AI model across the testing phase for 90-10% data division scenario.
ModelsR2MAPERMSEMAENSEWIDLNN0.970.052.241.630.970.99MARS0.840.095.203.900.840.96ELM0.820.125.784.370.800.95RF0.940.073.602.750.920.98Scatter plots were created to examine the linear relationship among the actual and predicted values for EF concrete compressive strength. The scatter plots for the three scenarios (70-30%, 80-20%, and 90-10%), and the developed AI models (DLNN, MARS, ELM, and RF) are depicted in Figures8–10. The 90-10% scenario achieved the best linear relationship between actual and predicted compressive strength values for the testing period using the DLNN model. Figure 10 indicates that the best prediction improvement is attained by the DLNN model with a maximum coefficient of determination R2 = 0.97.Figure 8
Scatter plot presentation using the developed AI model and for (70%-30%) data division scenario. (a) DLNN model. (b) MARS model. (c) ELM model. (d) RF model.
(a)(b)(c)(d)Figure 9
Scatter plot presentation using the developed AI model and for (80%-20%) data division scenario. (a) DLNN model. (b) MARS model. (c) ELM model. (d) RF model.
(a)(b)(c)(d)Figure 10
Scatter plot presentation using the developed AI model and for (90%-10%) data division scenario. (a) DLNN model. (b) MARS model. (c) ELM model. (d) RF model.
(a)(b)(c)(d)A two-dimensional graphical presentation named Taylor diagram is generated for investigating the performance for the developed AI model [85]. Taylor diagram is developed based on three statistical metrics correlation, RMSE, and standard deviation of the developed AI models as depicted in Figure 11. According to the location of the developed AI models on Taylor diagram, it can be noted that DLNN model achieved the best prediction performance for compressive strength of EF concrete by using a data division scenario of 90-10%.Figure 11
Taylor diagram for the developed AI model over all data division scenarios. (a) Scenario I. (b) Scenario II. (c) Scenario III.
(a)(b)(c)Another graphical presentation is created to give a good explanation for the performance of the developed AI models. The developed AI models were investigated by using relative error percentages for the three tested data division scenarios. Figure12 illustrates the relative error results of the developed model for scenarios 1, 2, and 3. According to the relative error results, the DLNN model attained the best predictive performance with a small relative error for scenario 3 with 90-10% data division.Figure 12
Relative error percentages of the developed AI models for the three tested scenarios. (a) Scenario I. (b) Scenario II. (c) Scenario III.
(a)(b)(c)Tables9–12 present the statistical measurement for integrating XGBoost combinations with the developed AI models over training and testing phase for scenario 3. The results indicated that DLNN and RF models performed better prediction than ELM and MARS models. The results also reported that the best results were achieved by including al input parameters in the modelling problem. The best prediction accuracy during training phase was attained by DLNN model with R2 = 0.97724, MAPE = 0.03867, RMSE = 2.0072, MAE = 1.37395, NSE = 0.97667, and WI = 0.99398. The testing phase results showed that DLNN model achieved highest prediction performance with R2 = 0.97106, MAPE = 0.04552, RMSE = 2.23648, MAE = 1.63199, NSE = 0.96952, and WI = 0.99160. From the reported results, it can be observed that compressive strength of concrete is affected by all input variables including cement, water, fine aggregate, recycled aggregate, age strength, and coarse aggregate.Table 9
Statistical measurements for DLNN model over training and testing phase for all input combinations.
ModelsR2MAPERMSEMAENSEWITraining phaseDLNN-model I0.670.147.525.590.670.89DLNN-model II0.820.115.644.290.820.95DLNN-model III0.820.115.574.190.820.95DLNN-model IV0.910.083.902.990.910.98DLNN-model V0.980.031.991.200.980.99DLNN-model VI0.980.042.011.370.980.99Testing phaseDLNN-model I0.680.147.335.450.670.87DLNN-model II0.880.094.433.520.880.97DLNN-model III0.900.084.073.160.900.97DLNN-model IV0.890.094.173.390.890.97DLNN-model V0.930.063.282.410.930.98DLNN-model VI0.970.052.241.630.970.99Table 10
Statistical measurements for MARS model over training and testing phase for all input combinations.
ModelsR2MAPERMSEMAENSEWITraining phaseMARS-model I0.430.197.525.590.670.89MARS-model II0.720.146.995.440.720.91MARS-model III0.770.136.364.980.770.93MARS-model IV0.800.125.884.660.800.94MARS-model V0.860.104.913.790.860.96MARS-model VI0.880.094.593.450.880.97Testing phaseMARS-model I0.350.2110.187.790.370.72MARS-model II0.750.126.334.910.760.92MARS-model III0.810.115.534.510.810.94MARS-model IV0.810.105.504.330.820.95MARS-model V0.840.105.234.200.830.96MARS-model VI0.840.095.203.900.840.96Table 11
Statistical measurements for ELM model over training and testing phase for all input combinations.
ModelsR2MAPERMSEMAENSEWITraining phaseELM-model I0.410.1910.137.600.410.74ELM-model II0.690.147.285.610.690.90ELM-model III0.760.126.414.960.760.93ELM-model IV0.830.115.434.180.830.95ELM-model V0.910.084.012.980.910.98ELM-model VI0.910.084.012.890.910.98Testing phaseELM-model I0.390.209.847.490.410.74ELM-model II0.700.136.945.520.710.91ELM-model III0.730.136.545.340.740.92ELM-model IV0.740.136.385.130.750.92ELM-model V0.780.126.134.680.770.94ELM-model VI0.820.125.784.370.800.95Table 12
Statistical measurements for RF model over training and testing phase for all input combinations.
ModelsR2MAPERMSEMAENSEWITraining phaseRF-model I0.640.157.895.830.640.89RF-model II0.740.136.745.130.740.92RF-model III0.740.136.725.140.740.92RF-model IV0.790.1336.784.860.790.94RF-model V0.890.084.483.280.880.97RF-model VI0.890.094.543.410.880.96Testing phaseRF-model I0.610.167.925.560.620.85RF-model II0.850.104.903.890.850.96RF-model III0.870.134.543.560.870.96RF-model IV0.870.104.643.890.870.96RF-model V0.900.074.042.910.900.97RF-model VI0.940.073.602.750.920.98Figure13 demonstrates the scatter plot diagram for all the developed model combinations. The DLNN model achieves the best results with highest value of the coefficient of determination R2 = 0.97. The diagram shows that the best prediction performance is achieved by using all influencing input variables. Applying all input variables increases prediction performance for all developed models (DLNN, MARS, ELM, RF).Figure 13
Scatter plot diagram for the developed AI models using all input combinations. (a) Model I. (b) Model II. (c) Model III. (d) Model IV. (e) Model V. (f) Model VI.
(a)(b)(c)(d)(e)(f)The Taylor diagram demonstrates the relationship between correlation, RMSE, and standard deviation for all input combinations, as shown in Figure14. Taylor’s presentation revealed that the developed AI models achieved good prediction using sixth combinations with all input parameters. DLNN model attained the best prediction performance using all input variables with the nearest position to the actual value.Figure 14
Taylor diagram for the developed AI models using all input combinations. (a) DLNN model. (b) MARS-model. (c) ELM-model. (d) RF-model.
(a)(b)(c)(d)Relative error presentation for the proposed AI models with all abstracted combinations is illustrated in Figure15. Graphical presentation of relative error percentages showed that modelling with six parameters attained low percentage error compared with the other combinations for all AI models. DLNN model with the sixth combination achieved the lowest relative error among the other three AI models (MARS, ELM, and RF).Figure 15
Relative error percentages of the developed AI models for the three tested scenarios. (a) DLNN model. (b) MARS-model. (c) ELM-model. (d) RF-model.
(a)(b)(c)(d)In the current study, a robust AI model was suggested for predicting the compressive strength for EF concrete. An advanced XGBoost algorithm was integrated with the developed AI model and the comparable once to investigate the best combinations of input parameters suitable for predicting EF concrete compressive strength. The results reported that the presented models realized an efficient prediction model that has the capability to handle the complex behaviour of concrete. This approach can improve concrete performance and the strength of EF concrete. Based on the reported results, it was observed that the size of data used for training AI model remarkably affects the accuracy of prediction performance. An accurate selection of training data size helps the modeller reduce an underfitting problem associated with the modelling process. The current study presented that 90% of training data is suitable to provide an accurate prediction of EF concrete compressive strength. The integration of XGBoost algorithm with AI models showed that an accurate prediction could be achieved by including all input parameters in the prediction problems. All six variables including cement, water, fine aggregate, recycled aggregate, age strength, and coarse aggregate are necessary to achieve good prediction accuracy of compressive strength. The results also reported that DLNN model provided an excellent ability to understand the nonlinearity between the contributed variables as well as the compressive strength of concrete. The statistical metrics and the graphical presentations established the robustness of the developed DLNN model for predicting the compressive strength of EF concrete. The proposed model indicated a robust machine learning model that can be implemented in actual practice to calculate the compressive strength of concrete as a prior stage for material design. This is giving credit to the application of computer-aided models to take place of an essential alternative technology. For future studies, the advanced models of AI can be introduced to improve the predictability of standalone AI models for compressive strength prediction of EF concrete such as [86, 87]. The application of uncertainty analysis can be examined to investigate the efficiency of input variables and the sensitivity of the developed model [88].


## 5. Conclusion

Accurate prediction of the compressive strength of EF concrete is very important in the sustainable development of concrete structures. Four AI approaches called DLNN, MARS, ELM, and RF were developed using 353 experiment samples collected from open-source literature. Three data division scenarios (70%-30%, 80%-20%, and 90%-10%) were applied to investigate the effect of changing training data size on the modelling process of compressive strength. XGBoost algorithm abstracted the most correlated input parameters needed to achieve the best prediction performance. The capacity of the developed model was examined by using statistical metrics and graphical presentations. The results reflected that the DLNN model attained the highest accuracy of prediction performance and, therefore, is suitable for the CS prediction of EF concrete. The results also demonstrated that using of 90%-10% data division scenario revealed a better prediction ability. It is found that the data size used for training the developed model has a considerable impact on the predicted model accuracy. The application of XGBoost as a feature selection approach plays an essential role in abstracting the most correlated variables in CS prediction. The study revealed that the highest prediction performance could be attained by incorporating all input variables in the prediction problem. The results reflected that the DLNN model attains the best prediction accuracy with the sixth input combination. In addition, the study demonstrated that the DLNN model provided an outstanding ability to capture the nonlinear relationship among the input variables as well as the compressive strength of concrete.



---

*Source: 5433474-2022-05-14.xml*