# A Novel Approach for Automatic Detection of Concrete Surface Voids Using Image Texture Analysis and History-Based Adaptive Differential Evolution Optimized Support Vector Machine

Authors: Nhat-Duc Hoang; Quoc-Lam Nguyen
Journal: Advances in Civil Engineering
 (2020)

---

## Abstract

To inspect the quality of concrete structures, surface voids or bugholes existing on a concrete surface after the casting process needs to be detected. To improve the productivity of the inspection work, this study develops a hybrid intelligence approach that combines image texture analysis, machine learning, and metaheuristic optimization. Image texture computations employ the Gabor filter and gray-level run lengths to characterize the condition of a concrete surface. Based on features of image texture, Support Vector Machines (SVM) establish a decision boundary that separates collected image samples into two categories of no surface void (negative class) and surface void (positive class). Furthermore, to assist the SVM model training phase, the state-of-the-art history-based adaptive differential evolution with linear population size reduction (L-SHADE) is utilized. The hybrid intelligence approach, named as L-SHADE-SVM-SVD, has been developed and complied in Visual C#.NET framework. Experiments with 1000 image samples show that the L-SHADE-SVM-SVD can obtain a high prediction accuracy of roughly 93%. Therefore, the newly developed model can be a promising alternative for construction inspectors in concrete quality assessment.


---

## Body

## 1. Research Background

The construction industry is widely known as a highly competitive environment within which product quality is a crucial element for a contractor’s survival [1]. In addition to the project cost and schedule, quality is a key factor that determines customer satisfaction [2]. Typically, for high-rise concrete buildings, architects and project owners impose strict specifications on the condition of the concrete surface. These requirements often involve the delivery of high-quality concrete surface with minimum presence of surface voids or bugholes [3]. It is because the phenomenon of excessive bugholes is one of the most serious and widely encountered defects on a formed concrete surface [4] and is often a subject of dispute between project owners, architects, and construction contractors [3].Surface voids (see Figure1) generally refer to small pits and craters on the concrete surface observed after the process of formwork removal [5]. These defects are brought about by the migration of an entrapped air bubble to the interface between fresh concrete and formwork [6]. The diameter of these voids typically ranges from several millimeters to 15 millimeters and even 25 millimeters in some cases [4]. For more details on the factors leading to the excessive presence of bugholes, readers are guided to the technical paper compiled by ACI [4]. In addition, bugholes are often distributed scatteredly on the concrete surface [7].Figure 1
Concrete surface voids.
(a)(b)(c)A high density of surface voids can result in several harmful effects on the performance of concrete structures:(i)
Bugholes obviously reduce the aesthetics of concrete structures(ii)
These voids reduce the protective depth of concrete structures and make the reinforcements inside them more vulnerable to corrosion [6](iii)
For structures retrofitted by fiber-reinforced plastic (FRP) material, excessive voids reduce the adhesion properties of the FRP material applied to the structure surface [8](iv)
Recent works have pointed out that salt accumulated in surface voids can lead to premature degradation of reinforced concrete structures [6, 9](v)
Existence of bugholes does increase the cost and time of painting and finishing activities because additional corrections must be performed to achieve a flat concrete surface [5]Consequently, project owners and architects demand that the number of surface voids on concrete surfaces should be limited to ensure the aesthetic appearance and durability of concrete structures. Currently, in Vietnam, as well as many other countries, inspection works on uncoated concrete surface are manually performed by human technicians. These procedures rely heavily on domain knowledge and the experience of inspectors with very limiting assistances of intelligent tools [4, 10–12]. Therefore, the current structure condition assessment process is notoriously time-consuming, laborious, and costly [7, 13–18]. It is due to the sheer volume of surface area needed to be inspected, labeled, and reported. Moreover, as pointed out by Perez et al. [19], the manual procedure also brings safety threat to human inspectors since certain concrete surfaces cannot be reached with ease, especially for those which are located at high/roof levels and narrow space. The current practice is also problematic due to the unavoidable inconsistency in inspecting outcomes. The reason is that the assessment process is dependent of subjective judgment of human technicians [2, 3].Therefore, project owners are increasingly seeking for fast, effective, and consistent tools to better structure condition assessment [19–24]. The assessment outcomes can also enhance communication between various stakeholders regarding the condition of the buildings. Image processing and machine learning with their fast pace of improvement provide feasible means to achieve such goals. Processed digital images and machine intelligence are capable of automating the concrete surface condition assessment, especially the task of detecting surface voids.Hence, in the recent years, various state-of-the-art methods relying on these two aforementioned techniques have been proposed to tackle the problem of interest. Zhu and Brilakis [25] put forward an image processing approach employing image segmentation and merging/splitting of pixels to detect air pockets on the concrete surfaces. Santos and Julio [26] presented an approach relying on the digital image processing and laser scanning techniques to analyze the roughness of the substrate surface. Fekri-Ershad and Tajeripour [27] put forward a robust approach based on a one-dimensional local binary pattern for recognizing surface defect. da Silva and Štemberk [28] also employed image processing (i.e., image binarization and morphological analyses) and fuzzy logic to inspect the surface quality of self-consolidating concrete for precast members with a focus on the presence of bugholes. Tajeripour and Fekri-Ershad [29] proposed novel one-dimensional local binary patterns used for recognizing abnormalities in stone textures.Kwasny et al. [30] investigated the influence of rheology on the quality of surface finish of cement-based mortars; the surface voids existing on concrete were then analyzed and quantified via digital image processing. Sadowski and Mathia [31] pointed out the needs of a more useful method for characterizing properties of a heterogeneous concrete surface; the authors also reviewed various image analyzing tools including image filtering and transformation. The effectiveness of wavelet transform and Gaussian image filtering in detecting surface defects were studied by Goïc et al. [32]. Liu and Yang [33] extracted the features of bugholes on a concrete surface via the utilization of the Otsu image binarization method. A texture classification model that incorporates of gray-level run-length matrix and robust illumination normalization techniques has been constructed by Dash and Senapati [34]. Khan et al. [35] relied on ground penetrating radar to detect water inside the cavities of concrete hollow core slabs.Yoshitake et al. [36] relied on binary image and color image analyses to detect bugholes distributed on sidewalls and tunnel-lining concrete. Perez et al. [19] utilized the state-of-the-art deep convolutional neural networks (DCNN) for detecting and categorizing building surface defects. A novel instance-level recognition and quantification for concrete surface bughole based on the deep neural network has been recently developed by Wei et al. [18]; this study demonstrates a great capability of machine learning in identifying concrete void surfaces accurately. Nevertheless, the implementation of deep learning models often requires a large number of training samples and a capable computing capability.As can be seen in the current literature, previous works have mainly relied on image thresholding methods for detecting concrete surface voids. These methods require substantial fine-tuning to adapt to variable characteristics of heterogeneous concrete surface [31]. Due to the complexity of concrete surface background and varying lighting conditions, the accuracy and applicability of image thresholding methods are generally limited. Therefore, image processing techniques should be integrated with advanced machine learning methods to deal with such issues. Hybrid image processing and machine learning tools have demonstrated their outstanding performances in detecting concrete surfaces in previous studies [15, 37–39]. However, too few studies have dedicated to investigating hybrid image processing—machine learning models for the problem of concrete surface void detection. Therefore, the current work is an attempt to fill this gap in the current literature.In this study, image processing techniques including the Gabor filtering and gray-level run lengths are employed to compute image texture of a concrete surface with and without voids. The texture information is, then, employed by the Support Vector Machines (SVM) to discriminate these two groups of concrete surface. Furthermore, since the training phase of the SVM-based bughole detection model requires a proper setting of the hyperparameters including the penalty coefficient and the kernel function parameter, the history-based adaptive differential evolution with linear population size reduction (L-SHADE) [40, 41] is used to optimize the model training phase. It is because these two hyperparameters strongly influence the learning and predicting performances of the SVM model. If the penalty coefficient and the kernel function parameter are not determined appropriately, the SVM-based surface void detection model cannot deliver the desired due to either overfitting or underfitting phenomena [42, 43].The subsequent parts of the study are organized in the following order: The second section reviews the research methodology including the employed image processing and computational tools, followed by the third section of the collected image samples; the fourth section presents the proposed hybrid model used for automatic detection of concrete surface voids. The fifth section reports experimental results and concluding remarks of this research are stated in the final section.


## 2. The Employed Image Processing and Computational Intelligence Methods

Since the surface of a concrete structure contains a diverse form of texture (e.g., intact surface, cracks, bugholes, and stains), texture information of an image region needs to be analyzed to support the surface void detection process. Accordingly, a large image is separated into a number of disjoint image samples of 20 × 20 pixels via image cropping operations. Subsequently, numerical features are extracted from these samples to construct the machine learning-based surface void recognition model. In this study, the Gabor filter and gray-level run-length methods are used for feature extraction. A novel method combining the SVM and L-SHADE is employed for data classification.


### 2.1. Gabor Filter (GF)

Gabor filters have been widely used for texture segmentation and feature extraction [44, 45]. Due to the capability of optimal joint localization in both spatial and spatial-frequency domains, Gabor filtering is an effective method for recognizing abnormal regions regular textured surfaces [46]. Various successful implementations of this texture discrimination approach have been reported in the literature [45–50]. Basically, a two-dimensional GF is a complex sinusoidal wave modulated by a Gaussian envelope [51]. This filter carries out a localized and oriented frequency analysis of a two-dimensional signal. Mathematically, the response of a GF can be given by the following equation [52]:(1)hx,yexp−12x2σx2+y2σy2cos2πu0x,where u0 denotes the frequency of a sinusoidal plane wave along the x-axis. σx and σy represent the space constants of the Gaussian envelope along the x- and y-axis, respectively.Notably, Gabor filters with different orientations can be attained via a rigid rotation of thex-y coordinate system [46]. The Fourier transform of the Gabor function described in equation (1) can be expressed as follows [52]:(2)Hu,v=Aexp−12u−u02σu2+v2σv2+exp−12u+u02σu2+v2σv2,where σu=1/2πσx, σv=1/2πσy, and A=2πσxσy. It is noted that this Fourier transform of the Gabor function determines the amount of each frequency component of the original image that is altered by a GF [50].Notably, to construct Gabor filters used for texture computation, their tuning parameters including the orientation angles and the radial frequency must be specified. As suggested in the previous work of Jain and Farrokhnia [52], 0°, 45°, 90°, and 135° orientations can be used. Moreover, given an image with a width of Nw pixels and Nw is a power of 2, the commonly used radial frequency u0 are as follows:12,22,42,…,Nw/42. Based on the response of the GF operations, statistical measurements can be calculated and employed as features for texture discrimination [53].


### 2.2. Gray-Level Run Lengths

In this study, the task of concrete surface voids is formulated as image texture discrimination. Therefore, information regarding a set of connected image pixels with their distinctive pattern needs to be analyzed. Due to the complex nature of concrete surface, automatic texture discrimination is by no means an easy task. To deal with such challenge, statistical texture analysis models can be employed. Among the statistical models, the gray-level run lengths (GLRL) [54] are very effective to extract information of an image sample based on sizes of homogeneous runs for each gray level [34].The GLRL was first proposed by Galloway [54] to distill discriminative features from terrain images. This method was, then, applied and improved by various scholars for classifying texture and other tasks in computer vision [34, 55]. This texture computation method relies on the fact that image texture can be considered as a pattern of gray intensity pixel in a particular direction from a reference point [34]. Based on the analysis of second-order statistical information, the GLRL computes the number of gray-level runs which is a collection of linearly adjacent pixels with similar gray intensities.Given an image sample and a certain direction, a run-length matrixp(i, j) stores the information regarding the number of times that the sample contains a run length j of gray level i [54]. Based on p(i, j) with different orientations (e.g. 0°,45°,90°,135°), a variety of texture information can be obtained [56].Based on a constructed run-length matrix, the Short Run Emphasis (SRE), Long Run Emphasis (LRE), Gray-Level Nonuniformity (GLN), Run-Length Nonuniformity (RLN), and Run Percentage (RP) are computed according to the following equations [54, 57, 58]:(3)SRE=1Nr∑i=1M∑j=1Npi,jj2,LRE=1Nr∑i=1M∑j=1Npi,j×j2,GLN=1Nr∑i=1M∑j=1Npi,j2,RLN=1Nr∑j=1N∑i=1Mpi,j2,RP=NrNp,where M and N are the number of gray levels and the maximum run length. Nr is the total number of runs, and Np denotes the number of pixels in the image.Furthermore, Chu et al. [55] extended the original GLRL’s measurement with the indices of Low Gray-Level Run Emphasis (LGRE) and High Gray-Level Run Emphasis (HGRE):(4)LGRE=1Nr∑j=1N∑i=1Mpi,ji2,HGRE=1Nr∑j=1N∑i=1Mpi,j×i2.Additionally, Short Run Low Gray-Level Emphasis (SRLGE), Short Run High Gray-Level Emphasis (SRHGE), Long Run Low Gray-Level Emphasis (LRLGE), and Long Run High Gray-Level Emphasis (LRHGE) are put forward by Dasarathy and Holder [59]:(5)SRLGE=1Nr∑j=1N∑i=1Mpi,ji2×j2,SRHGE=1Nr∑j=1N∑i=1Mpi,j×i2j2,LRLGE=1Nr∑j=1N∑i=1Mpi,j×j2i2,LRHGE=1Nr∑j=1N∑i=1Mpi,j×i2×j2.


### 2.3. The History-Based Adaptive Differential Evolution with Linear Population Size Reduction

Differential Evolution (DE) [60, 61] is unquestionably a powerful stochastic search for solving numerical optimization. This stochastic search engine relies on a novel integrated mutation-crossover operation to explore and exploit the search space. During the last decade, various enhancements of the standard DE have been put forward to improve its searching performance [62–65].Among these enhanced DE variants, the history-based adaptive differential evolution with linear population size reduction (L-SHADE) [40, 41] stands out to be a highly successful version with competitive outcomes reported in various comparative studies [66–69]. Therefore, this improved version of the DE is selected in this study to optimize the performance of the machine learning and image processing-based concrete void detection model.The L-SHADE algorithm, proposed by Tanabe and Fukunaga [41], improves the original DE algorithm via several aspects. First, the mutation scale (F) and the crossover probability (CR) are fine-tuned adaptively during the optimization process instead of being fixed. Second, an effective mutation strategy called DE/current-to-pbest/1 is implemented to better explore the search space. Third, a population size shrinking strategy is used to both enhance the convergence rate and reduce computational expense.The operational flow of the L-SHADE algorithm is tersely presented in Figure2. Since the L-SHADE inherits the main characteristics of the standard DE, its searching process can also be divided into four steps of population initialization, mutation, crossover, and selection. In the first step, based on the prespecified searched domain (lower and upper boundaries), the number of decision variable (NDV), and an initial number of members (NM), a population of NM vectors is randomly created and expected to be distributed evenly throughout the searched space. In the second step, a new candidate solution called a mutated vector is generated via the DE/current-to-pbest/1 strategy. The DE/current-to-pbest/1 strategy used for generating a new candidate solution is given by [40](6)vi,g+1=xi,g+Fixr1,g−xr2,g+Fixpbest,g−xi,g.Figure 2
The operational flow of the L-SHADE algorithm.In the third step, the crossover operation is used to combine the information of the newly created candidate and its parent according to the following manner [60]:(7)uj,i,g+1=vj,i,g+1,ifrandj≤Crorj=rnbi,xj,i,g,ifrandj>Crandj≠rnbi.In the last step, a greedy selection which compares the fitness of the candidate solution and its parent is carried out. It is noted that the L-SHADE employs archives of MF and MCR which are vectors of a fixed lengthH; these two archives store the mean values of the mutation scale and the crossover probability. Moreover, the two sets of SF and SCR store all CR and F values that helped to yield child solutions better than the parents. After each generation, the current population size reduces via the removal of inferior solutions [41].


### 2.4. Support Vector Machine

Support Vector Machines (SVM), constructed on the basis of the statistical learning theory, are a robust method for establishing pattern classification models. Introduced by Vapnik [70], the SVM have gained popularity in the research community via various works which reported their successful implementations [71–73]. It is because this machine learning method features significant advantages including resilience to noisy data via a framework of maximum margin construction and capability of handling nonlinearly separable data by means of kernel tricks. Furthermore, the learning phase of the SVM is bolt down to solving a convex optimization problem; this guarantees a global convergence and avoids being trapped in local optima [74].The concept of the SVM used for concrete surface void detection is demonstrated in Figure3. The model deals with nonlinearly separable data by mapping the data from the original input space to a high-dimensional feature space; in such high-dimensional feature space, linear separability can become feasible.Figure 3
Demonstration of an SVM model.Given a training datasetxk,ykk=1N with a numerical feature xk∈Rn and corresponding class categories yk∈−1,+1, an SVM model establishes a classification boundary to distinguish data from a positive class +1 (surface void) and a negative class −1 (nonsurface void). It is noted that a numerical feature xk is actually texture information extracted from an image sample using the Gabor filter and the GLRL. To establish such classification boundary, it is required to solve the following nonlinear programming problem [43]:(8)MinimizeJpw,e=12wTw+c12∑k=1Nek2Subjected toykwTφxk+b≥1−ek,k=1,…,N,ek≥0,where w∈Rn is a normal vector to the classification hyperplane and b∈R represents the model bias; ek>0 is slack variables; c represents a penalty constant; and φx denotes the nonlinear data mapping used for dealing with nonlinear separable data.It is noted that the SVM does not necessitate an explicit expression of the data mapping functionφx. The quantity of interest is the product of φx in the input space which is defined as a Kernel function:(9)Kxk,xl=φxkTφxl.The Radial Basis Function Kernel (RBFK) is often employed and it is shown as follows:(10)Kxk,xl=exp−xk−xl22σ2,where σ represents a tuning parameter of the RBFK.After solving the aforementioned nonlinear programming, the SVM model used for data classification can be tersely presented in the following equation:(11)yxl=sign∑k=1SVαkykKxk,xl+b,where αk denotes the solution of the dual form of the aforementioned nonlinear programming. SV represents the number of support vectors (the number of αk>0).


## 2.1. Gabor Filter (GF)

Gabor filters have been widely used for texture segmentation and feature extraction [44, 45]. Due to the capability of optimal joint localization in both spatial and spatial-frequency domains, Gabor filtering is an effective method for recognizing abnormal regions regular textured surfaces [46]. Various successful implementations of this texture discrimination approach have been reported in the literature [45–50]. Basically, a two-dimensional GF is a complex sinusoidal wave modulated by a Gaussian envelope [51]. This filter carries out a localized and oriented frequency analysis of a two-dimensional signal. Mathematically, the response of a GF can be given by the following equation [52]:(1)hx,yexp−12x2σx2+y2σy2cos2πu0x,where u0 denotes the frequency of a sinusoidal plane wave along the x-axis. σx and σy represent the space constants of the Gaussian envelope along the x- and y-axis, respectively.Notably, Gabor filters with different orientations can be attained via a rigid rotation of thex-y coordinate system [46]. The Fourier transform of the Gabor function described in equation (1) can be expressed as follows [52]:(2)Hu,v=Aexp−12u−u02σu2+v2σv2+exp−12u+u02σu2+v2σv2,where σu=1/2πσx, σv=1/2πσy, and A=2πσxσy. It is noted that this Fourier transform of the Gabor function determines the amount of each frequency component of the original image that is altered by a GF [50].Notably, to construct Gabor filters used for texture computation, their tuning parameters including the orientation angles and the radial frequency must be specified. As suggested in the previous work of Jain and Farrokhnia [52], 0°, 45°, 90°, and 135° orientations can be used. Moreover, given an image with a width of Nw pixels and Nw is a power of 2, the commonly used radial frequency u0 are as follows:12,22,42,…,Nw/42. Based on the response of the GF operations, statistical measurements can be calculated and employed as features for texture discrimination [53].


## 2.2. Gray-Level Run Lengths

In this study, the task of concrete surface voids is formulated as image texture discrimination. Therefore, information regarding a set of connected image pixels with their distinctive pattern needs to be analyzed. Due to the complex nature of concrete surface, automatic texture discrimination is by no means an easy task. To deal with such challenge, statistical texture analysis models can be employed. Among the statistical models, the gray-level run lengths (GLRL) [54] are very effective to extract information of an image sample based on sizes of homogeneous runs for each gray level [34].The GLRL was first proposed by Galloway [54] to distill discriminative features from terrain images. This method was, then, applied and improved by various scholars for classifying texture and other tasks in computer vision [34, 55]. This texture computation method relies on the fact that image texture can be considered as a pattern of gray intensity pixel in a particular direction from a reference point [34]. Based on the analysis of second-order statistical information, the GLRL computes the number of gray-level runs which is a collection of linearly adjacent pixels with similar gray intensities.Given an image sample and a certain direction, a run-length matrixp(i, j) stores the information regarding the number of times that the sample contains a run length j of gray level i [54]. Based on p(i, j) with different orientations (e.g. 0°,45°,90°,135°), a variety of texture information can be obtained [56].Based on a constructed run-length matrix, the Short Run Emphasis (SRE), Long Run Emphasis (LRE), Gray-Level Nonuniformity (GLN), Run-Length Nonuniformity (RLN), and Run Percentage (RP) are computed according to the following equations [54, 57, 58]:(3)SRE=1Nr∑i=1M∑j=1Npi,jj2,LRE=1Nr∑i=1M∑j=1Npi,j×j2,GLN=1Nr∑i=1M∑j=1Npi,j2,RLN=1Nr∑j=1N∑i=1Mpi,j2,RP=NrNp,where M and N are the number of gray levels and the maximum run length. Nr is the total number of runs, and Np denotes the number of pixels in the image.Furthermore, Chu et al. [55] extended the original GLRL’s measurement with the indices of Low Gray-Level Run Emphasis (LGRE) and High Gray-Level Run Emphasis (HGRE):(4)LGRE=1Nr∑j=1N∑i=1Mpi,ji2,HGRE=1Nr∑j=1N∑i=1Mpi,j×i2.Additionally, Short Run Low Gray-Level Emphasis (SRLGE), Short Run High Gray-Level Emphasis (SRHGE), Long Run Low Gray-Level Emphasis (LRLGE), and Long Run High Gray-Level Emphasis (LRHGE) are put forward by Dasarathy and Holder [59]:(5)SRLGE=1Nr∑j=1N∑i=1Mpi,ji2×j2,SRHGE=1Nr∑j=1N∑i=1Mpi,j×i2j2,LRLGE=1Nr∑j=1N∑i=1Mpi,j×j2i2,LRHGE=1Nr∑j=1N∑i=1Mpi,j×i2×j2.


## 2.3. The History-Based Adaptive Differential Evolution with Linear Population Size Reduction

Differential Evolution (DE) [60, 61] is unquestionably a powerful stochastic search for solving numerical optimization. This stochastic search engine relies on a novel integrated mutation-crossover operation to explore and exploit the search space. During the last decade, various enhancements of the standard DE have been put forward to improve its searching performance [62–65].Among these enhanced DE variants, the history-based adaptive differential evolution with linear population size reduction (L-SHADE) [40, 41] stands out to be a highly successful version with competitive outcomes reported in various comparative studies [66–69]. Therefore, this improved version of the DE is selected in this study to optimize the performance of the machine learning and image processing-based concrete void detection model.The L-SHADE algorithm, proposed by Tanabe and Fukunaga [41], improves the original DE algorithm via several aspects. First, the mutation scale (F) and the crossover probability (CR) are fine-tuned adaptively during the optimization process instead of being fixed. Second, an effective mutation strategy called DE/current-to-pbest/1 is implemented to better explore the search space. Third, a population size shrinking strategy is used to both enhance the convergence rate and reduce computational expense.The operational flow of the L-SHADE algorithm is tersely presented in Figure2. Since the L-SHADE inherits the main characteristics of the standard DE, its searching process can also be divided into four steps of population initialization, mutation, crossover, and selection. In the first step, based on the prespecified searched domain (lower and upper boundaries), the number of decision variable (NDV), and an initial number of members (NM), a population of NM vectors is randomly created and expected to be distributed evenly throughout the searched space. In the second step, a new candidate solution called a mutated vector is generated via the DE/current-to-pbest/1 strategy. The DE/current-to-pbest/1 strategy used for generating a new candidate solution is given by [40](6)vi,g+1=xi,g+Fixr1,g−xr2,g+Fixpbest,g−xi,g.Figure 2
The operational flow of the L-SHADE algorithm.In the third step, the crossover operation is used to combine the information of the newly created candidate and its parent according to the following manner [60]:(7)uj,i,g+1=vj,i,g+1,ifrandj≤Crorj=rnbi,xj,i,g,ifrandj>Crandj≠rnbi.In the last step, a greedy selection which compares the fitness of the candidate solution and its parent is carried out. It is noted that the L-SHADE employs archives of MF and MCR which are vectors of a fixed lengthH; these two archives store the mean values of the mutation scale and the crossover probability. Moreover, the two sets of SF and SCR store all CR and F values that helped to yield child solutions better than the parents. After each generation, the current population size reduces via the removal of inferior solutions [41].


## 2.4. Support Vector Machine

Support Vector Machines (SVM), constructed on the basis of the statistical learning theory, are a robust method for establishing pattern classification models. Introduced by Vapnik [70], the SVM have gained popularity in the research community via various works which reported their successful implementations [71–73]. It is because this machine learning method features significant advantages including resilience to noisy data via a framework of maximum margin construction and capability of handling nonlinearly separable data by means of kernel tricks. Furthermore, the learning phase of the SVM is bolt down to solving a convex optimization problem; this guarantees a global convergence and avoids being trapped in local optima [74].The concept of the SVM used for concrete surface void detection is demonstrated in Figure3. The model deals with nonlinearly separable data by mapping the data from the original input space to a high-dimensional feature space; in such high-dimensional feature space, linear separability can become feasible.Figure 3
Demonstration of an SVM model.Given a training datasetxk,ykk=1N with a numerical feature xk∈Rn and corresponding class categories yk∈−1,+1, an SVM model establishes a classification boundary to distinguish data from a positive class +1 (surface void) and a negative class −1 (nonsurface void). It is noted that a numerical feature xk is actually texture information extracted from an image sample using the Gabor filter and the GLRL. To establish such classification boundary, it is required to solve the following nonlinear programming problem [43]:(8)MinimizeJpw,e=12wTw+c12∑k=1Nek2Subjected toykwTφxk+b≥1−ek,k=1,…,N,ek≥0,where w∈Rn is a normal vector to the classification hyperplane and b∈R represents the model bias; ek>0 is slack variables; c represents a penalty constant; and φx denotes the nonlinear data mapping used for dealing with nonlinear separable data.It is noted that the SVM does not necessitate an explicit expression of the data mapping functionφx. The quantity of interest is the product of φx in the input space which is defined as a Kernel function:(9)Kxk,xl=φxkTφxl.The Radial Basis Function Kernel (RBFK) is often employed and it is shown as follows:(10)Kxk,xl=exp−xk−xl22σ2,where σ represents a tuning parameter of the RBFK.After solving the aforementioned nonlinear programming, the SVM model used for data classification can be tersely presented in the following equation:(11)yxl=sign∑k=1SVαkykKxk,xl+b,where αk denotes the solution of the dual form of the aforementioned nonlinear programming. SV represents the number of support vectors (the number of αk>0).


## 3. The Image Samples of the Concrete Surface

To construct the SVM machine learning model used for concrete surface void recognition, the set of image samples capturing the texture of concrete structures must be prepared. This image set includes samples which contain concrete bugholes and samples without such defect. Accordingly, a set of 1000 image samples with assigned ground truth categories has been manually collected via field trip to several construction sites in Danang city (Vietnam). To guarantee a balanced dataset, the numbers of the negative (without surface voids) and positive (having surface voids) samples are both 500. The categories of image samples have been determined by human inspectors. Herein, the label = -1 means the negative class and the label = 1 denotes a positive class. It is noted that the collected images in this study have been taken by using the Cannon EOS M10 (CMOS 18.0 MP) and Nikon D5100 (CMOS 16.2 MP). To enhance the speed of the texture computation phase and to ensure the consistency of an image region, the image size has been set to be 20x20 pixels. The image samples are illustrated in Figure4. Additionally, to better cope with the diversity of the concrete surface, the negative class of nonsurface void deliberately includes samples of sound concrete surface, cracks, and stains.Figure 4
The collected image samples: (a) nonsurface void and (b) surface void.
(a)(b)


## 4. The Proposed Hybridization of Image Processing, Machine Learning, and Metaheuristic for Detecting Concrete Surface Voids

The proposed hybridization of image processing, machine learning, and metaheuristic optimization used for detection of concrete surface voids is presented in this section of the study. The integrated model is denoted as L-SHADE-SVM-SVD. The overall model structure is graphically summarized in Figure5. The L-SHADE-SVM-SVD model is developed in the Visual C#.NET environment (Framework 4.6.2) and run in ASUS FX705GE—EW165T (Core i7 8750H, 8 GB Ram, 256 GB solid-state drive).Figure 5
The proposed L-SHADE-SVM-SVD used for automatic detection of concrete surface voids.The L-SHADE-SVM-SVD operation can be divided into three steps:(i)
Image texture computation: the step computes texture information of concrete surface obtained from image samples stored in training and testing subsets. The first subset includes 90% of the collected samples and is used for model construction. The second set occupies 10% of the collected samples and is reserved for validating the model predictive capability. Image texture including the Gabor filter and GLRL is computed and used as the numerical feature.(ii)
The L-SHADE metaheuristic optimization: as mentioned earlier, the model training and pattern classification phases of the SVM require appropriate values of the penalty coefficient (c) and the kernel function parameter (σ). The former hyperparameter dictates how the loss function of the SVM increases due to misclassified data points. The latter hyperparameter affects the smoothness of the decision boundary. Therefore, these hyperparameters strongly influence the learning and predictive performance of the SVM-based bughole detection model. The selection of the penalty coefficient (c) and the kernel function parameter (σ) can be formulated as an optimization task within which metaheuristic algorithms can be employed. Based on the previous comparative works [41, 67, 68, 75], this study employs the L-SHADE metaheuristic algorithm for conducting the SVR model optimization. This DE variant first generates an initial population of hyperparameters in a random manner. In each generation, the optimization algorithm explores and exploits the search space to gradually guide the population to a better solution representing SVM models with good predictive capability.(iii)
The SVM-based pattern classification: based on the optimized solution of the model hyperparameters, the SVM model is employed to construct a decision surface that is capable of distinguish concrete surface with and without voids. Notably, the SVM model is constructed via a built-in function supported by the Accord.NET Framework [76].It is also noted that, to optimize the SVM model performance, aK-fold cross validation (with K = 5) is used. Using this cross validation, the whole dataset is separated into 5 mutual exclusive subsets. In each of the five runs, one subset is used for model testing and the other subsets are employed for model training. The average predictive performance is used to quantify the model generalization capability. Accordingly, the following cost function is used by the L-SHADE-SVM-SVD:(12)CF=∑k=1KFNRk+FPRkK,where FNRk and FPRk denote the false negative rate (FNR) and the false positive rate (FPR) obtained from kth run, respectively.The FNR and FPR indices are computed as follows:(13)FNR=FNFN+TP,FPR=FPFP+TN,where FN, FP, TP, and TN are false negative, false positive, true positive, and true negative data samples, respectively.Moreover, it is noted that, to compute the Gabor filter, it is necessary to convert the original image to grayscale ones. Based on the recommendations of Jain and Farrokhnia [52], four values of orientations (0°, 45°, 90°, and 135°) and four values of radial frequency (12,22,42,82) have been employed to calculate the Gabor filtering features. In this study, the Gabor filters are implemented with the assistance of built-in functions provided by the Accord.NET Framework [76]. Based on the filtered image, the following statistical indices can be measured to characterize image texture:(i)
The mean of the Gabor filter response:(14)MeanGaborFilter=∑i=0HIM−1∑j=0WIM−1GFRi,jWIM×HIM,where HIM and WIM are the width and height of an image sample, respectively. GFR(i, j) denotes the Gabor filter response at a pixel (i, j).(ii)
The standard deviation of the Gabor filter response:(15)STDGaborFilter=∑i=0HIM−1∑j=0WIM−1GFRi,j−MeanGaborFilter2WIM×HIM.(iii)
The skewness of the Gabor filter response [77]:(16)SkewnessGaborFilter=1/WIM×HIM∑i=0HIM−1∑j=0WIM−1GFRi,j−MeanGaborFilter31/WIM×HIM−1∑i=0H−1∑j=0W−1GFRi,j−MeanGaborFilter23/2.(iv)
The entropy of the Gabor filter response:(17)EntropyGaborFilter=−∑i=0NDV−1FOHFilter×log2PFilter,where FOHFilter represents the first-order histogram of the Gabor filter response. Moreover, NDV = 256 denotes the number of discrete gray intensity values for an 8 bit grayscale image.Because 16 Gabor filters are employed for computing image texture and each filter has four statistical indices (the mean, standard deviation, skewness, and entropy), the number of Gabor filtering features is 64. Moreover, since there are four orientations (0°,45°,90°,and135°) used for GLRL matrix construction and each GLRL matrix has 11 features, the number of GLRL features is 44 [57]. Accordingly, the total number of features used for the SVM pattern classification is 64 + 44 = 108.Additionally, to facilitate the data classification based on the SVM, the established dataset has been normalized by the Z-score equation given by(18)XZN=Xo−mXsX,where Xo and XZN represent the original and the standardized input feature, respectively. mX and sX denote the mean and the standard deviation of the original input feature, respectively.


## 5. Experimental Result and Comparison

As mentioned earlier, to train and validate the hybrid model used for concrete surface void detection, the collected dataset has been divided into two packages of training and testing datasets. The training dataset (90% of the original dataset) is used for model construction, and the testing dataset is reserved for evaluating the model performance when predicting novel image samples. Moreover, to diminish the effect of randomness in data sampling and to reliably assess the predictive capability of the newly developed method, the training/testing data sampling has been performed 20 times. In each time of running, 10% of the original dataset is randomly drawn out to form the testing dataset; the rest of the original dataset is used for model construction. The datasets used for time of model running are illustrated in Table1. Herein, the column of sample index expresses the number of data samples employed in the training and testing datasets.Table 1
The training and testing datasets.
DatasetsSample indexFeaturesClass labelF1F2F3…F106F107F108Training1108.11533.580−15.236…6866.94611.141658.625−12109.64533.763−18.768…5026.96511.007381.245−1………………………899106.06332.555−14.066…32167.85410.600640.7011900108.30533.135−13.449…31379.52410.6741039.1981Testing1108.43532.954−16.630…9425.74411.8022016.670−12105.56530.844−16.665…8383.50411.5051440.862−1………………………99107.22832.628−12.087…30558.25910.551631.6241100107.22332.418−13.006…30292.41710.575514.3721In addition, to quantify the predictive capability of the proposed L-SHADE-SVM-SVD, the classification accuracy rate (CAR), precision, recall, negative predictive value (NPV), and F1 score are computed from the four basic results of TP, TN, FP, and FN. These performance indicators are presented as follows [78]:(19)CAR=TP+TNTP+TN+FP+FN×100%,Precision=TPTP+FP,Recall=TPTP+FN,NPV=TNTN+FN,F1Score=2TP2TP+FP+FN.The L-SHADE with an initial population size of 30 and a maximum number of searching generations of 100 was utilized to seek for the most appropriate set of the SVM model’s hyperparameters. The searching progress of the L-SHADE metaheuristic is graphically presented in Figure6. The optimization process yields the penalty parameter (c) = 3.712 and the kernel function parameter (σ) = 6.031. In addition, the statistical outcomes of the L-SHADE-SVM-SVD (mean and standard deviation) using the hyperparameters found by the L-SHADE metaheuristic are reported in Table 2. The averaging values of the CAR, precision, recall, NPV, and F1 score are 92.600%, 0.911, 0.942, 0.940, and 0.926, respectively. In addition, the average runtime of the proposed approach is roughly 13.30 seconds. Since there are 100 data samples in the testing set, the computational time used for classifying one testing data sample is approximately 0.13 seconds.Figure 6
The L-SHADE optimization process.Table 2
Prediction performance of the L-SHADE-SVM-SVD.
PhasesIndicesCAR (%)TPTNFPFNPrecisionRecallNPVF1 scoreTrainingMean97.417437.200439.5509.55013.7000.9790.9700.9700.974Std.0.2224.9385.7531.6051.3020.0040.0030.0030.002TestingMean92.60046.20046.4004.5002.9000.9110.9420.9400.926Std.2.7614.8735.3051.8782.1250.0370.0400.0440.027Furthermore, to demonstrate the predictive ability of the newly developed L-SHADE-SVM-SVD model used for concrete surface void recognition, the model performance has been compared to those of Deep Convolution Neural Network (DCNN) models [79–81] with stochastic gradient descent with momentum (Sgdm) [82], Adaptive Moment Estimation (Adam) [83], and root mean square propagation (Rmsprop) [84]. The three DCNN models are denoted as DCNN-Sgdm, DCNN-Adam, and DCNN-Rmsprop and are implemented via the MATLAB deep learning toolbox [85]. Via a trial-and-error process, appropriate configurations of the DCCN model are selected and shown in Table 3.Table 3
Configurations of the DCNN models.
CNN layersConvolution layersPooling layersFilter numberFilter sizeFilter size136102236823364243622Moreover, the minibatch Backpropagation Neural Network (MB-BPNN) [86, 87] is also employed as a benchmark method. The MB-BPNN model has been developed in Visual C#.NET by the authors and trained with the minibatch mode [87, 88]; the batch-size is selected to be 32. and the number of neurons in the hidden layer is set to be 2/3DX+CN, as suggested by Heaton [89]; DX and CN denote the numbers of features and outputs, respectively. The MB-BPNN model is, then, trained with the sigmoidal activation function with the maximum number of epochs = 1000 epochs and the learning rate = 0.01.The prediction results of the proposed L-SHADE-SVM-SVD model, as well as the benchmark models, are summarized in Table4 and graphically presented as box plots in Figure 7. As can be seen from the prediction results, the performance of the L-SHADE-SVM-SVD (CAR = 92.600%, Precision = 0.911, Recall = 0.942, NPV = 0.940, and F1 score = 0.926) is better than that of the DCNN-Rmsprop (CAR = 88.350%,Precision = 0.929,Recall = 0.834, NPV = 0.852,andF1score = 0.877),DCNN-Adam (CAR = 86.900%,Precision = 0.893, Recall = 0.844, NPV = 0.856, and F1 score = 0.865),DCNN-Sgdm(CAR = 86.800%,Precision = 0.909, Recall = 0.818, NPV = 0.839, and F1 score = 0.859),and MB-BPNN (CAR = 85.700%, Precision = 0.818, Recall = 0.890, NPV = 0.896, and F1 score = 0.839).Table 4
Prediction result comparison.
PhaseIndicesL-SHADE-SVM-SVDDCNN-RmspropDCNN-AdamDCNN-SgdmMB-BPNNMeanStd.MeanStd.MeanStd.MeanStd.MeanStd.TrainingCAR (%)97.4170.22287.9222.58589.2111.93787.7566.16586.7636.889TP437.2004.938369.00025.984390.25017.693375.65040.359333.90063.570TN439.5505.753422.30020.683412.65021.755414.15021.866360.2009.540FP9.5501.60527.70020.68337.35021.75535.85021.86666.10063.570FN13.7001.30281.00025.98459.75017.69374.35040.35939.8009.540Precision0.9790.0040.9340.0440.9160.0420.9120.0640.8350.159Recall0.9700.0030.8200.0580.8670.0390.8350.0900.8950.012NPV0.9700.0030.8420.0410.8750.0290.8510.0650.9010.024F1 score0.9740.0020.8710.0300.8890.0190.8700.0750.8520.132TestingCAR (%)92.6002.76188.3503.13386.9004.20486.8006.17885.7007.248TP46.2004.87341.7003.24642.2003.79240.9004.76740.9008.130TN46.4005.30546.6502.32344.7003.32645.9002.67444.8002.587FP4.5001.8783.3502.3235.3003.3264.1002.6749.1008.130FN2.9002.1258.3003.2467.8003.7929.1004.7675.2002.587Precision0.9110.0370.9290.0450.8930.0570.9090.0650.8180.163Recall0.9420.0400.8340.0650.8440.0760.8180.0950.8900.042NPV0.9400.0440.8520.0490.8560.0590.8390.0670.8960.052F1 score0.9260.0270.8770.0360.8650.0440.8590.0760.8390.137Figure 7
Box plots of model performances.In addition, the two-samplet-test [90] is utilized in this study to better confirm the statistical significance of the model predictive capabilities. This statistical test is often employed to inspect the null hypothesis that the model prediction performances of two machine learning models may be drawn from normal distributions with equal means. Herein, the significant level (p value) of the test is set to be 0.05, and the results of the t-test are reported in Table 5. As can be observed from this table, the p values <0.05 reliably reject the null hypothesis. This fact confirms that the proposed L-SHADE-SVM-SVD is best suited for the task of detecting concrete surface bugholes.Table 5
Thet-test outcomes of pairwise model comparisons.
Model comparisonTest outcomep valueL-SHADE-SVM-SVD vs. DBNN-RmspropSignificant0.00005L-SHADE-SVM-SVD vs. DBNN-AdamSignificant0.00001L-SHADE-SVM-SVD vs. DBNN-SgdmSignificant0.00046L-SHADE-SVM-SVD vs. MB-BPNNSignificant0.00030


## 6. Conclusions

Detection of a concrete surface is crucial for inspecting quality of cast-in-place concrete elements. To enhance the productivity and eliminate subjective judgment of concrete quality inspection works, this research proposes an intelligent method that hybridizes state-of-the-art image processing, machine learning, and metaheuristic methods. Image texture is used as the input feature that characterizes the states of a concrete surface. Gabor filter and GLRL-based texture information is computed and employed by the SVM to construct a decision boundary that divides the input into two categories of no surface void (negative class) and surface void (positive class).In addition, to optimize the SVM model training phase, the L-SHADE metaheuristic is used. This metaheuristic algorithm autonomously searched for the most appropriate set of the SVM model’s hyperparameters including the penalty coefficient and the kernel function parameter. The integrated model, named as L-SHADE-SVM-SVD, has been developed and compiled in the Visual C#.NET framework to ease its implementation. The experimental outcome using 1000 image samples and a repeated data sampling with 20 runs demonstrate that the newly developed L-SHADE-SVM-SVD is able to attain good predictive performances (CAR = 92.600%, Precision = 0.911, Recall = 0.942, NPV = 0.940, and F1 score = 0.926). Thus, the L-SHADE-SVM-SVD can be a helpful tool to assist construction inspectors in assessing concrete surface quality.Nevertheless, since the current L-SHADE-SVM-SVD model aims at recognizing the status of no surface void and surface void, the task of localizing surface voids on a concrete surface image can be performed in a future study. In addition, other future directions of the current works may include the extension of the collected image dataset to enhance the generalization of the developed machine learning model, investigation of other advanced metaheuristic algorithms to improve the surface void detection performance, and employment of other performance measurements such as model runtime [91].



---

*Source: 4190682-2020-07-28.xml*