# Fabric Defect Detection in Textile Manufacturing: A Survey of the State of the Art

Authors: Chao Li; Jun Li; Yafei Li; Lingmin He; Xiaokang Fu; Jingjing Chen
Journal: Security and Communication Networks
 (2021)

---

## Abstract

Defects in the textile manufacturing process lead to a great waste of resources and further affect the quality of textile products. Automated quality guarantee of textile fabric materials is one of the most important and demanding computer vision tasks in textile smart manufacturing. This survey presents a thorough overview of algorithms for fabric defect detection. First, this review briefly introduces the importance and inevitability of fabric defect detection towards the era of manufacturing of artificial intelligence. Second, defect detection methods are categorized into traditional algorithms and learning-based algorithms, and traditional algorithms are further categorized into statistical, structural, spectral, and model-based algorithms. The learning-based algorithms are further divided into conventional machine learning algorithms and deep learning algorithms which are very popular recently. A systematic literature review on these methods is present. Thirdly, the deployments of fabric defect detection algorithms are discussed in this study. This paper provides a reference for researchers and engineers on fabric defect detection in textile manufacturing.


---

## Body

## 1. Introduction

The influence of artificial intelligence on industrial field has far exceeded our expectations [1, 2]. The vast number of researchers and engineers are constantly accelerating the development of industrial intelligence. In 2010, Germany put forward the Industry 4.0 framework, and the framework has been promoted and applied widely among the European Union member states. Subsequently, the United States and China put forward corresponding plans and policies for smart manufacturing of their own country. Industry 4.0 is the inevitable trend of the future development of manufacturing industry [3].Industrial artificial intelligence is typical cross-disciplinary, which combines knowledge of mechanical, data science, network, communication, information security, and other disciplines, and it aims to use artificial intelligence algorithms to solve industrial problems and improve the efficiency and security of manufacturing [4, 5]. Towards industry 4.0, the textile manufacturing industry also needs to find its own way to adapt the manufacturing process. Textile manufacturing is a large-scale and complicated industry. The textile manufacturing process consists of a series of complex and orderly processes, mainly including spinning, weaving, dyeing, printing and finishing, and garments manufacturing. The stability and quality of the textile fabric produced by the whole production line are crucial to any enterprise [4].There are many factors that affect the final product on the production line of textile manufacturing, such as material quality, mechanical factors, dye type, yarn size, and human factors [5]. In general, textile fabric defects refer to defects on the surface of the fabric. There are many types of fabric defects, most of which are caused by process problems and machine malfunctions. Defects will affect the quality of the final product, resulting in a great waste of all kinds of resources [6, 7]. In the process of fabric manufacturing, the defects in the previous stage will affect the later stage. Therefore, early detection of fabric defects can reduce the loss of enterprises earlier and faster [8]. Therefore, effective fabric defect detection is one of the key measures for modern fabric manufacturers to control cost and enhance product value and core competence.In modern textile manufacturing, automatic fabric defect is an important way to ensure the textile quality [9]. For long, fabric defect detection is implemented by manual visual inspection which is inadequate and expensive in the meantime. Accordingly, automatic fabric defect detection is necessary for the textile industry to reduce cost and increase productivity. The core of a complete online textile fabric defect detection system is the detection algorithms. Many researchers and engineers in this field have devoted themselves to the design of robust and efficient algorithms within the past few decades [10]. Compared to manual fabric defects detection, the automatic detection systems are more effective with higher efficiency. Fabric defect detection has been a hot research field in computer science and technology and mechanical engineering. This paper provides not only the algorithms for the researchers but also the deployment problems for the practitioners [11].This review summarizes and classifies the methods of fabric defect detection in a broader scope. A total of more than 2,000 articles were retrieved, from which 128 articles were included in this review. The main search terms used for retrieval are “Fabric defect detection,” “textile inspection,” “fabric defect recognition,” “automatic textile Manufacturing,” etc. In addition to the traditional and classical methods mentioned in the previous reviews, this review discusses and compares the algorithms of deep learning algorithms in defect detection which are very popular in recent years. Finally, the deployment of the algorithm is also discussed; for the algorithm, the deployment is also very important to the accuracy and efficiency of the system implementation. It is hoped that this review will provide some help and suggestions for the application of AI in fabric manufacturing.


## 2. Fabric Defect Detection Methods

Fabric defect detection algorithms are roughly divided into two categories in this study, traditional algorithms and learning-based algorithms, as shown in Figure1. Most of the traditional algorithms are based on feature engineering with prior knowledge, covering statistical, structural, spectral, and model-based methods. The learning-based algorithms can be further divided into classical machine learning algorithms and deep learning algorithms. Machine learning uses mathematical algorithms to learn and analyze data to make predictions and take decisions in the future, which has been widely employed in recent years and achieved stratifying results in various disciplines and industries.Figure 1
An overview of the defect detection methods covered in this review.


### 2.1. Traditional Algorithms

#### 2.1.1. Statistical Algorithms

Statistical approaches utilize the spatial distribution of gray values in images [12], such as gray-level co-occurrence matrices (GLCM), autocorrelation analysis, and fractal dimension features.Raheja et al. present an automated fabric defect detection system utilizing GLCM. In this approach, a signal graph is constructed with GLMC statistics and interpixel distance. Then a comparison between nondefective image and test image is made. Additionally, a Gabor filter based approach is utilized to detect the defects in this study. The conclusion is made that GLCM based algorithms generate higher detection accuracies and less computational complexity [13, 14].Anandan et al. [15] combine the GLCM and curvelet transform (CT) by extracting the eigenvector of the defect which makes the fabric defect features more evident. The experiments show the effectiveness of the proposed algorithm with comparison to GLCM and wavelet-based methods, respectively.Kumar et al. [16] design a statistical approach for identifying defects in fabric images using eigenvalues. Using the coefficient of variation, defective portions of the fabric images are identified. This method is simple and easy to use according to the experiments in the work.Intending to effectively detect fabric defects, Song et al. [17] calculate the membership degree of each fabric region. Utilizing the extreme point density map of the image combined with the features of the membership function region, the saliency of defect regions is obtained. The whole scheme further adopts a threshold method and morphological processing. The author states this algorithm can detect fabric defects efficiently and accurately in the presence of noise and background texture interference.Gharsallah et al. [18] present a fabric defect detection approach utilizing an improved anisotropic diffusion filter and saliency image features. Given that the conventional anisotropic diffusion methods cannot identify the defect edge which is confused with the background texture, the improved anisotropic diffusion method combines the local gradient magnitude with a saliency map. This approach can effectively remove the texture background and retain the image defect edge.Table1lists several statistical algorithms used for textile fabric defect detection in brief.Table 1
Statistical algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSayed [19]Entropy filtering and minimum error thresholdingTILDA datasetDetection success rateKumari [20]Sylvester matrix based similarity methodKTH-TIPS-I and KTH-TIPS-IIFalse positives and false negativesChetverikov and Hanbury [21]Based on two fundamental structural properties, regularity and local orientation (anisotropy)Brodatz images and TILDA datasetDetection success rate


#### 2.1.2. Spectral Approach

Fourier transform, Gabor transform, wavelet transform, and discrete cosine transform [22–24] are the representative method of spectral methods. These algorithms mentioned in this survey are listed in Table 2. Fourier transform, wavelet transform, and Gabor transform-based methods have been thoroughly studied and tested on fabric defect detection applications.Table 2
Spectral algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSulochan [25]Multiscale wavelet features and fuzzy C-means clusteringReal and computer-simulated fabric imagesDetection error rateVermaak et al. [26]Dual-tree complex wavelet transform (DTCWT)TILDA datasetDetection success rateLiu and Zheng [27]The method based on information entropy and frequency domain saliencyDatabase created by the research associate of the industrial automation research laboratoryACC, true positive rate(TPR), false positive rate(FPR), positive predictive value (PPV), negative predictive value (NPV), time, F-measureDi et al. [28]L0 gradient minimization method and two-dimensional fractional Fourier transform (2D-FRFT) for obtaining the saliency map of the quaternion imageDataset from automation laboratory fabric database of Hong Kong UniversityTrue positive (TP), false positive (FP), true negative (TN), and false negative (FN)Jing [29]Gabor preprocessed golden image subtractionIndustrial automation laboratory at the University of Hong Kong and the TILD databaseDetection success rateMohammed and Alhamdani [30]Fuzzy back propagation neural network (FBPNN) with Gabor featuresCollected datasetDetection success rateYapi et al. [31]Using learning-based local textural distributions in the contourlet domainTILDA database(TP, FP, TN, and FN) local precision (PL), local recall (RL), and local accuracy (ACCL)Li et al. [32] propose an algorithm employing a multiscale wavelet transform and Gaussian mixture model for automated fabric defect detection. A textile fabric image was decomposed by the “Pyramid” wavelet transform and then reconstructed using thresholding method. Next, the Gaussian mixture model was utilized to segment the reconstructed image. The experiments demonstrate the effectiveness of the proposed algorithm for detecting and segmenting the defect images.Rebhi et al. [33] present a fabric defect detection approach using local homogeneity information and discrete cosine transform (DCT). DCT was applied to the newly calculated homogeneity image and different energy features of all DCT blocks are then extracted. And the extracted features are fed into the feedforward neural networks classifier.


#### 2.1.3. Structural Approach

One effective way for segmenting defective area on the patterned textile fabric image is the golden image subtraction (GIS) method. Ngan et al. proposed a method named wavelet preprocessed golden image subtraction (WGIS) [34]. Additionally, wavelet transforms, GIS, and WGIS methods are compared in this study, and the proposed WGIS achieved the best performance among them.Jia and Liang [35] segment the fabric images into nonoverlapping regions named lattices and then the similarity of these lattices are calculated in the feature space. The proposed Isotropic Lattice Segmentation (ILS) method shows satisfying results on the box and star pattern image database. Jia et al. brought up another approach in their later study [36] on the basis of lattice segmentation and lattice templates. In this study, the lattices are segmented according to different placement rules of texture primitives that belong to different classes. The distances of undetermined lattice and lattice template are calculated, and the lattices are regarded as the defective area when the distances are larger than a certain threshold. The algorithm is further improved by adding template statistics which are learned from defect-free images in [37].Another template-based correction approach for fabric images with periodical structure is introduced in Chang’s work [38]. A fabric image is divided into lattices due to variation regularity and correction is then made to reduce the lattice misalignment. The defective lattices are first located and defect regions are segmented at a later step. Based on its assumptions, the lattice segmentation and template-based correction algorithms are more suitable for patterned fabric images.Shi et al. [39] propose a method using low-rank decomposition of gradient information combined with structured graph. The fabric image is first divided into defect-free regions and defect regions based on the structured graph information. Adaptive thresholding is utilized during the lattice merging step. Finally, the matrix decomposition is calculated under the prior information from the segmentation results; thus the defect regions are emphasized. The presented method outperforms other methods on a standard database.Abouelela et al. [40] design a fabric defect detection system employing simple statistical features such as median, mean, and variance. The author holds that time efficiency is crucial to any industrial procedure. Therefore, the author exchanged the complexity of simple features calculation for real-time performance. The proposed algorithm is better at detecting defects that vary drastically in the physical dimension.


#### 2.1.4. Model-Based Methods

Ngan et al. propose motif-based methods for detecting defects in 2D patterned texture. This kind of methods is based on the assumption that patterned images can be divided into lattices and motifs. And further energy of moving subtraction is calculated to differentiate defective and defect-free region [41]. In order to reduce the detection rate of false positives and false negatives, the Gaussian mixture model is used to represent the energy variance value [42]. K-means clustering is applied to the data and the convex hull of each cluster is calculated in that fitting ellipsoidal region.Lucia et al. propose an algorithm for detecting the fabric defects in uniformly structured textile fabric images. The algorithm includes two stages: the feature extraction stage and the defect recognition stage. In the first stage, the symmetric Gabor filter bank and principal component analysis are utilized for feature extraction, and in the second stage, the Euclidean norm of features is calculated and compared for defect recognition. This algorithm is designed on a patch basis and proved to be effective on the public TILDA database [43].For environment-friendly textiles, Shu et al. [44] adopt an algorithm based on principal component analysis and nonlocal average filtering to enhance the fabric texture and reduce the noise. A texture-based defect measurement method is used for calculating the similarity; thus defect and nondefect areas can be distinguished.Some researchers treat the fabric defect detection problem as a one-class classification. Bu et al. [45, 46] propose a method based on the support vector data description (SVDD) model. In the training stage, the multiple fractal features are extracted and the optimal parameters are selected for the Gaussian kernel function. The detection results on several datasets demonstrate that this combination is effective. Bu et al. [47] also present another method using SVDD model, and the features extracted in this method are based on autoregressive (AR) spectral estimation model combined with Burg algorithm.With analysis on statistical model-based methods, Campbell et al. [48] propose two model-based methods in their study. The first method states the maximum likelihood of image binarization. The other method is mainly for defect detection in repeated weaving patterns; thus the discrete Fourier transformation is utilized for texture analysis. In the end, a model-based clustering method is applied to delineate the defective regions.For the detection of patterned fabrics, Tsang et al. [49] propose a method named Elo rating (ER) in line with the spirit of sports. The fabric images are divided into standard-size partitions. Matches are calculated between partitions and revised through an Elo point matrix. The defect area (partition) will win the competition as a powerful player. This presented method was tested on dot-, star-, and box-patterned fabrics database and obtained comparable results to the most advanced method.


### 2.2. Learning-Based Algorithms

#### 2.2.1. Classical Machine Learning Algorithms

(1) Dictionary Learning-Based Algorithms. Many researchers have validated the effectiveness of dictionary learning-based algorithms dealing with textile fabric defect detection problems [50, 51]. The general steps of these algorithms: first a dictionary is learned from the training or test image, and then a fabric image without defects is reconstructed using the learned dictionary; thenceforth the detection is implemented by subtracting the reconstructed image from the test image. Recently, many algorithms based on low-rank representation have been brought up in the application of fabric defect inspection. In order to solve the optimization problem of the objective function, many methods reduce the low-rank decomposition problem to the nuclear minimization (NNM) problem.Li et al. propose an algorithm on the basis of biological vision modeling. The biological visual saliency is modeled by low-rank representation (LRR); thus the fabric image is decomposed into salient defect regions and defect-free backgrounds [52].Li et al. [53] model a defect-free region as a low-rank structure and the defect region as a sparse structure. Thus a fabric image can be regarded as the sum of a low-rank matrix and a sparse matrix. For dimensionality reduction, instead of singular value decomposition (SVD) on the matrix of the original image, the presented method uses eigenvalue decomposition on blocked image matrix. Therefore, this method is easy to implement and works well given that the contrast of the fabric image is sufficient.Table3tabulates some other dictionary learning-based algorithms using low-rank decomposition.Table 3
Dictionary learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationLi et al. [52]Low-rank representation (LRR)(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryPrecision and recallLi et al. [53]Low-rank representation500 fabric images from the textile kind C1 of the TILDA database(a) Sensitivity and specificity; (b) false alarm rate (FAR), missing rate (MR)Gao et al. [54]Gabor filter and tensor low-rank recoveryDataset from the research associate of industrial automation research laboratoryReceiver operating characteristic curve (ROC)Shi et al. [39, 47]Low-rank decomposition with gradient informationDataset from the research associate of industrial automation research laboratoryTPR, FPR, PPV, NPVLiu et al. [55–57]Multi-scale convolutional neural network and low-rank decomposition model(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryMeans and standard deviations of average precisions, recalls, F-measure, and mean absolute error (MAE)Mo et al. [58]Weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weightsDatabase is from the research associate of industrial automation research laboratory, HKBUVisual defect locating results, the metrics of false alarm, recall, precision, accuracy, and F-measureLi et al. [59]Low-rank decomposition of multichannel feature matrices(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryROC curves and precision-recall (PR) curvesYang et al. [60]Sparse and dense mixed low-rank decompositionReal-world samples of 512∗512 with 256-gray levelsSaliency map (qualitative)Wang et al. [61]A randomized low-rank and sparse matrix decomposition model named GoDecFabric image dataset collected by Dr. Henry Y. T. Ngan [62]Precision, recall, and F-measureShi et al. [39] point out two shortcomings of the low-rank decomposition. One is that existing low-rank decomposition models barely detect the defect regions with high gradients. And the other shortcoming is that small defect area or complex area will be incorrectly segmented given the inaccuracy of prior information. To overcome these shortcomings, Shi et al. propose a low-rank decomposition method utilizing gradient information combined with structured graph algorithm. The proposed method outperforms others on the point, box, and star databases.Traditional machine learning algorithms, such as KNN [63] and neural network [64], are widely used in fabric defection detection problems. And feature engineering is one of the major processes in the machine learning life cycle.Mak et al. extracted [65] four novel fractal features and employed support vector data description (SVDD), which is a support vector machine learning algorithm used for one-class classification, for fabric defect detection in his work.Zhang et al. propose an approach employing the radial basis function (RBF) network. Gaussian mixture model (GMM) is utilized to improve the accuracy of Gaussian RBF parameter estimation. The validity of the proposed method has been proved on multiple class datasets [66].Tian and Li [67] propose an autoencoder-based method for fabric defect detection by exploring similarities between image patches. Utilizing the repeated texture pattern, similar nondefective patches were found for each candidate defect patch and the corresponding latent variables were weighted and combined according to which the original latent variable can be modified. Experimental results manifest the effectiveness of the proposed algorithm.Yapi et al. [68] consider this problem as a binary problem. A compact and accurate feature set was extracted by statistical modeling of multiscale contourlet decomposition, and then a Bayesian classifier (BC) is used to classify the defect and nondefect classes. This algorithm obtained high precision detection with real-time efficiency.Some other traditional machine learning algorithms for fabric defect detection are shown in Table4.Table 4
Traditional machine learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationWang et al. [63]Multiview stereo vision (MVS) and bag-of-features (BOF), K-nearest neighbor (KNN) algorithmCollected datasetDetection success ratePriyanka and Manish [69]Artificial neural networks (ANN)Collected datasetDetection success rateBumrungkun [70]Snake active contour and support vector machinesCollected datasetRecognition accuracy detection success rateZhang et al. [71]L0 gradient minimization (LGM) and the fuzzy c-means (FCM) method to detect various fabric defects with diverse texturesImages from the automation laboratory sample database of Hong Kong University, TILDA textile texture database, and Guang Dong Esquel TextilesACC, TPR, FPR, PPV, and IOU (intersection over union)


#### 2.2.2. Deep Learning Algorithms

Recently, many researchers have applied deep learning techniques to fabric defect detection problems and have achieved satisfying results [72, 73] for the improvement of textile product quality and production efficiency [74]. Although deep learning methods have been proved to be powerful when dealing with segmentation and classification problems, there are still some problems in the practical application of specific industries [75]. First of all, the actual textile production line requires high real-time performance of the algorithm, which is the demand of high execution efficiency. Furthermore, compared to normal defect-free samples, the defective image data is difficult to obtain, which brings challenges to the training process of deep learning [76].At present, the deep learning-based object detector can be classified as one-stage detectors and two-stage detectors [77]. Classical deep learning algorithms for object detection are listed in Table 5. In general, one-stage detectors have fast detection speed to meet the requirements of online detection, but the detection accuracy usually fails to meet requirements. In contrast, the two-stage algorithms have higher detection accuracy, but its detection speed is difficult to meet the real-time requirements of the algorithm in production scenes. In the field of fabric defect detection, the advantages and disadvantages of one-stage and two-stage detection algorithms are quite similar to those in other fields. The two-stage algorithm has higher accuracy but slower speed than the one-stage algorithm. In the actual application of textile industry, we hope that, under the premise of satisfying the detection accuracy, the faster the detection speed, the better. Therefore, the algorithm should be selected according to the actual application scenarios and requirements to find the balance between efficiency and accuracy.Table 5
Deep learning algorithms for detecting object.
One-stage detectorsTwo-stage detectorsYOLOFaster RCNNSSDMask RCNNYOLOv2/v3/v4Cascade RCNNRefineDetFPNRetinaNetR-FCN(1) One-Stage Detection Algorithms. One-stage detection algorithm does not have a separate proposal generation phase. Typically, these algorithms treat all locations on the image as potential objects and manage to categorize each interest region into a target object or background.The recently proposed single shot multibox detector (SSD) is a typical one-stage detector that has obtained good detection performance in object detection. This algorithm is designed based on a convolutional neural network (CNN). Some improvements have been made for the fabrics defect scenario by Liu et al. [78] and the experimental results show rationality and effectiveness.Ouyang et al. [79] present a CNN based algorithm for on-loom fabric defect inspection. This proposed algorithm introduces a dynamic activation layer utilizing the defect probability information with a pairwise potential function to a CNN. This algorithm obtains good results dealing with the unbalanced data classification problem.Deep convolutional neural network (DCNN) based algorithms have achieved satisfactory results on visual tasks and have been widely used in industrial scenarios. Liu et al. [80] employ DCNN to detect fabric defects with complicated textures. This proposed approach is particularly designed for real textile production environment with limited resources. A series of improvements have been done to make the detection more effective. Zhou et al. propose an efficient DCNN architecture focusing on the problem of fabric defect detection, called Efficient Defect Detectors (EDDs) [81]. To extract more low-level features, EDDs adjust the input resolution, depth, and width using a scaling strategy. The improvement proved to be effective when compared with existing fabric defect detection algorithms.Xu et al. [82] propose a novel detection network named de-deformation defect detection network (D4Net). This model is composed of reference generation, de-deformation network, and marginal loss. The most suitable reference is selected and paired with the input image and then is sent into the de-deformation network. The dissimilarities are calculated and enhanced by the marginal loss. Experiments on a large industrial database containing 67K images have been done and the results show that the algorithm outperforms other algorithms especially for large pattern fabric images.Peng et al. put forward a detection algorithm called Priori Anchor Convolutional Neural Network (PRAN-Net) to fix this problem. Feature Pyramid Network (FPN) is utilized to selected multiscale feature maps and then sparse priori anchors are generated based on ground truth boxes [83].Li et al. [84] propose an architecture using several microarchitectures. The microarchitecture is constructed of multiscale analysis, filter factorization, multilocation pooling, and parameters reduction, thus making the network a compact one. With the small model size, the proposed net worked well on fabric defect detection.(2) Two-Stage Detection Algorithms. With regard to two-stage detectors, a sparse set of proposals is generated in the first stage and in the second stage, and the features of generated proposals are sent into DCNN for prediction results. As a remarkable two-stage detector, Faster R-CNN is an object detection model that improves on Fast R-CNN by utilizing a region proposal network (RPN) with the CNN model. Some researchers utilize the modified Faster R-CNN model for fabric defect detection [85–87]. Jun et al. [88] propose a framework that utilized the Inception-V1 model and LeNet-5 model. This approach includes local defect prediction in the first stage and global defect recognition in the second stage.Table6 lists some other deep learning algorithms utilized in textile fabric defect applications.Table 6
Deep learning-based algorithms for fabric defect detection.
AuthorProposed or tested modelCategorizeDatasetEvaluationJing et al. [89]Mobile-UnetOne-stageBenchmark databases, the fabric images database (FID), and yarn dyed fabric images (YFI), in which all images are manually annotated segmentationPixel accuracy (PA) and IoUHong-wei hang [90]YOLOV2One-stageCollected dataset (276)IOU, recall, and precisionYoung-Joo Han [91]Stacked convolutional autoencodersOne-stageSynthetic and collected datasetRecall, precision, and F-scoreXinying He [92]Adaptive method based on DenseNet-SSDOne-stageCollected dataset (2072)Calculate localization loss (loc) and confidence loss (conf)Mohammed et al. [93]A multilayer perceptron with a Levenberg–Marquardt (LM) algorithmOne-stageCollected dataset (217)Specificity, accuracy, and sensitivityShuang mei [94]Multiscale convolutional denoising autoencoder network modelOne-stageFour datasets: fabrics, KTH-TIPS, Kylberg texture, and ms-textureRecall, precision, and F1-measureHuosheng Xie [95]Improved RefineDetOne-stageTILDA dataset, Hong Kong patterned textures database, and DAGM2007 datasetPrecision (P), recall I, F1-score, mean average precision (mAP), model parameter (param.)Yanqing Huang [96]Segmentation network and decision networkTwo-stageDark redfFabric (DRF), light blue fabric (LBF) and patterned texture fabric (PTF)Frames per second (FPS) Avg-IoU and Avg-PIn recent years, Generative Adversarial Networks (GANs) have attracted a lot of attention [97, 98]. GANs and related algorithms have been widely used in a range of computer vision and computer graphics applications, such as image synthesis and video generation. GAN based fabric defect detection algorithms can automatically adapt to different fabric textures by learning existing fabric defect samples [99]. Liu et al. design a deep semantic segmentation network to detect fabric defects. They train a multistage GAN model to synthesize reasonable defect samples from nondefect samples. The performance of the method was verified by comprehensive experiments on all sorts of typical fabric image samples.Le et al. [100] utilize Wasserstein generative adversarial nets (WGANs) combined with transfer learning techniques and multimodel ensembling framework. The effectiveness of the proposed scheme is demonstrated on unbalanced and rare datasets of images with defects.Inspired by biological visual perception mechanism, Zhao et al. [101] describe a CNN model based on visual long-short-term memory (VLSTM). Three types of features, visual perception features, visual short-term memory (VSTM) features, and visual long-term memory (VLTM) features, are extracted by stacked convolutional autoencoders, a shallow CNN, and nonlocal neural networks, respectively. Experiments have been done on three public datasets and results show that the proposed algorithm is comparable to state-of-the-art algorithms.Traditional saliency detection models usually rely on hand-crafted features to capture the information of global context and local details. Researchers add the attention mechanism to deal with fabric defect detection problems. Wang et al. [102] propose a deep saliency detection model that incorporated self-attention mechanism into a CNN for fabric defect detection. Multiscale feature maps are generated from a fully convolutional network, and a self-attention module is used to coordinate the dependence between the features of different layers. The self-attention mechanism in this algorithm proved to be very effective with complex or blurred defects.Some studies combine the traditional and deep learning methods. Wang et al. [103] extracted global deep features using CNN in combination with handcrafted low-level features, and nonconvex robust PCA regularized by nonconvex total variation are employed to data processing and noise reduction. Then a segmentation algorithm is used to segment the saliency map to obtain the defect area.


## 2.1. Traditional Algorithms

### 2.1.1. Statistical Algorithms

Statistical approaches utilize the spatial distribution of gray values in images [12], such as gray-level co-occurrence matrices (GLCM), autocorrelation analysis, and fractal dimension features.Raheja et al. present an automated fabric defect detection system utilizing GLCM. In this approach, a signal graph is constructed with GLMC statistics and interpixel distance. Then a comparison between nondefective image and test image is made. Additionally, a Gabor filter based approach is utilized to detect the defects in this study. The conclusion is made that GLCM based algorithms generate higher detection accuracies and less computational complexity [13, 14].Anandan et al. [15] combine the GLCM and curvelet transform (CT) by extracting the eigenvector of the defect which makes the fabric defect features more evident. The experiments show the effectiveness of the proposed algorithm with comparison to GLCM and wavelet-based methods, respectively.Kumar et al. [16] design a statistical approach for identifying defects in fabric images using eigenvalues. Using the coefficient of variation, defective portions of the fabric images are identified. This method is simple and easy to use according to the experiments in the work.Intending to effectively detect fabric defects, Song et al. [17] calculate the membership degree of each fabric region. Utilizing the extreme point density map of the image combined with the features of the membership function region, the saliency of defect regions is obtained. The whole scheme further adopts a threshold method and morphological processing. The author states this algorithm can detect fabric defects efficiently and accurately in the presence of noise and background texture interference.Gharsallah et al. [18] present a fabric defect detection approach utilizing an improved anisotropic diffusion filter and saliency image features. Given that the conventional anisotropic diffusion methods cannot identify the defect edge which is confused with the background texture, the improved anisotropic diffusion method combines the local gradient magnitude with a saliency map. This approach can effectively remove the texture background and retain the image defect edge.Table1lists several statistical algorithms used for textile fabric defect detection in brief.Table 1
Statistical algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSayed [19]Entropy filtering and minimum error thresholdingTILDA datasetDetection success rateKumari [20]Sylvester matrix based similarity methodKTH-TIPS-I and KTH-TIPS-IIFalse positives and false negativesChetverikov and Hanbury [21]Based on two fundamental structural properties, regularity and local orientation (anisotropy)Brodatz images and TILDA datasetDetection success rate


### 2.1.2. Spectral Approach

Fourier transform, Gabor transform, wavelet transform, and discrete cosine transform [22–24] are the representative method of spectral methods. These algorithms mentioned in this survey are listed in Table 2. Fourier transform, wavelet transform, and Gabor transform-based methods have been thoroughly studied and tested on fabric defect detection applications.Table 2
Spectral algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSulochan [25]Multiscale wavelet features and fuzzy C-means clusteringReal and computer-simulated fabric imagesDetection error rateVermaak et al. [26]Dual-tree complex wavelet transform (DTCWT)TILDA datasetDetection success rateLiu and Zheng [27]The method based on information entropy and frequency domain saliencyDatabase created by the research associate of the industrial automation research laboratoryACC, true positive rate(TPR), false positive rate(FPR), positive predictive value (PPV), negative predictive value (NPV), time, F-measureDi et al. [28]L0 gradient minimization method and two-dimensional fractional Fourier transform (2D-FRFT) for obtaining the saliency map of the quaternion imageDataset from automation laboratory fabric database of Hong Kong UniversityTrue positive (TP), false positive (FP), true negative (TN), and false negative (FN)Jing [29]Gabor preprocessed golden image subtractionIndustrial automation laboratory at the University of Hong Kong and the TILD databaseDetection success rateMohammed and Alhamdani [30]Fuzzy back propagation neural network (FBPNN) with Gabor featuresCollected datasetDetection success rateYapi et al. [31]Using learning-based local textural distributions in the contourlet domainTILDA database(TP, FP, TN, and FN) local precision (PL), local recall (RL), and local accuracy (ACCL)Li et al. [32] propose an algorithm employing a multiscale wavelet transform and Gaussian mixture model for automated fabric defect detection. A textile fabric image was decomposed by the “Pyramid” wavelet transform and then reconstructed using thresholding method. Next, the Gaussian mixture model was utilized to segment the reconstructed image. The experiments demonstrate the effectiveness of the proposed algorithm for detecting and segmenting the defect images.Rebhi et al. [33] present a fabric defect detection approach using local homogeneity information and discrete cosine transform (DCT). DCT was applied to the newly calculated homogeneity image and different energy features of all DCT blocks are then extracted. And the extracted features are fed into the feedforward neural networks classifier.


### 2.1.3. Structural Approach

One effective way for segmenting defective area on the patterned textile fabric image is the golden image subtraction (GIS) method. Ngan et al. proposed a method named wavelet preprocessed golden image subtraction (WGIS) [34]. Additionally, wavelet transforms, GIS, and WGIS methods are compared in this study, and the proposed WGIS achieved the best performance among them.Jia and Liang [35] segment the fabric images into nonoverlapping regions named lattices and then the similarity of these lattices are calculated in the feature space. The proposed Isotropic Lattice Segmentation (ILS) method shows satisfying results on the box and star pattern image database. Jia et al. brought up another approach in their later study [36] on the basis of lattice segmentation and lattice templates. In this study, the lattices are segmented according to different placement rules of texture primitives that belong to different classes. The distances of undetermined lattice and lattice template are calculated, and the lattices are regarded as the defective area when the distances are larger than a certain threshold. The algorithm is further improved by adding template statistics which are learned from defect-free images in [37].Another template-based correction approach for fabric images with periodical structure is introduced in Chang’s work [38]. A fabric image is divided into lattices due to variation regularity and correction is then made to reduce the lattice misalignment. The defective lattices are first located and defect regions are segmented at a later step. Based on its assumptions, the lattice segmentation and template-based correction algorithms are more suitable for patterned fabric images.Shi et al. [39] propose a method using low-rank decomposition of gradient information combined with structured graph. The fabric image is first divided into defect-free regions and defect regions based on the structured graph information. Adaptive thresholding is utilized during the lattice merging step. Finally, the matrix decomposition is calculated under the prior information from the segmentation results; thus the defect regions are emphasized. The presented method outperforms other methods on a standard database.Abouelela et al. [40] design a fabric defect detection system employing simple statistical features such as median, mean, and variance. The author holds that time efficiency is crucial to any industrial procedure. Therefore, the author exchanged the complexity of simple features calculation for real-time performance. The proposed algorithm is better at detecting defects that vary drastically in the physical dimension.


### 2.1.4. Model-Based Methods

Ngan et al. propose motif-based methods for detecting defects in 2D patterned texture. This kind of methods is based on the assumption that patterned images can be divided into lattices and motifs. And further energy of moving subtraction is calculated to differentiate defective and defect-free region [41]. In order to reduce the detection rate of false positives and false negatives, the Gaussian mixture model is used to represent the energy variance value [42]. K-means clustering is applied to the data and the convex hull of each cluster is calculated in that fitting ellipsoidal region.Lucia et al. propose an algorithm for detecting the fabric defects in uniformly structured textile fabric images. The algorithm includes two stages: the feature extraction stage and the defect recognition stage. In the first stage, the symmetric Gabor filter bank and principal component analysis are utilized for feature extraction, and in the second stage, the Euclidean norm of features is calculated and compared for defect recognition. This algorithm is designed on a patch basis and proved to be effective on the public TILDA database [43].For environment-friendly textiles, Shu et al. [44] adopt an algorithm based on principal component analysis and nonlocal average filtering to enhance the fabric texture and reduce the noise. A texture-based defect measurement method is used for calculating the similarity; thus defect and nondefect areas can be distinguished.Some researchers treat the fabric defect detection problem as a one-class classification. Bu et al. [45, 46] propose a method based on the support vector data description (SVDD) model. In the training stage, the multiple fractal features are extracted and the optimal parameters are selected for the Gaussian kernel function. The detection results on several datasets demonstrate that this combination is effective. Bu et al. [47] also present another method using SVDD model, and the features extracted in this method are based on autoregressive (AR) spectral estimation model combined with Burg algorithm.With analysis on statistical model-based methods, Campbell et al. [48] propose two model-based methods in their study. The first method states the maximum likelihood of image binarization. The other method is mainly for defect detection in repeated weaving patterns; thus the discrete Fourier transformation is utilized for texture analysis. In the end, a model-based clustering method is applied to delineate the defective regions.For the detection of patterned fabrics, Tsang et al. [49] propose a method named Elo rating (ER) in line with the spirit of sports. The fabric images are divided into standard-size partitions. Matches are calculated between partitions and revised through an Elo point matrix. The defect area (partition) will win the competition as a powerful player. This presented method was tested on dot-, star-, and box-patterned fabrics database and obtained comparable results to the most advanced method.


## 2.1.1. Statistical Algorithms

Statistical approaches utilize the spatial distribution of gray values in images [12], such as gray-level co-occurrence matrices (GLCM), autocorrelation analysis, and fractal dimension features.Raheja et al. present an automated fabric defect detection system utilizing GLCM. In this approach, a signal graph is constructed with GLMC statistics and interpixel distance. Then a comparison between nondefective image and test image is made. Additionally, a Gabor filter based approach is utilized to detect the defects in this study. The conclusion is made that GLCM based algorithms generate higher detection accuracies and less computational complexity [13, 14].Anandan et al. [15] combine the GLCM and curvelet transform (CT) by extracting the eigenvector of the defect which makes the fabric defect features more evident. The experiments show the effectiveness of the proposed algorithm with comparison to GLCM and wavelet-based methods, respectively.Kumar et al. [16] design a statistical approach for identifying defects in fabric images using eigenvalues. Using the coefficient of variation, defective portions of the fabric images are identified. This method is simple and easy to use according to the experiments in the work.Intending to effectively detect fabric defects, Song et al. [17] calculate the membership degree of each fabric region. Utilizing the extreme point density map of the image combined with the features of the membership function region, the saliency of defect regions is obtained. The whole scheme further adopts a threshold method and morphological processing. The author states this algorithm can detect fabric defects efficiently and accurately in the presence of noise and background texture interference.Gharsallah et al. [18] present a fabric defect detection approach utilizing an improved anisotropic diffusion filter and saliency image features. Given that the conventional anisotropic diffusion methods cannot identify the defect edge which is confused with the background texture, the improved anisotropic diffusion method combines the local gradient magnitude with a saliency map. This approach can effectively remove the texture background and retain the image defect edge.Table1lists several statistical algorithms used for textile fabric defect detection in brief.Table 1
Statistical algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSayed [19]Entropy filtering and minimum error thresholdingTILDA datasetDetection success rateKumari [20]Sylvester matrix based similarity methodKTH-TIPS-I and KTH-TIPS-IIFalse positives and false negativesChetverikov and Hanbury [21]Based on two fundamental structural properties, regularity and local orientation (anisotropy)Brodatz images and TILDA datasetDetection success rate


## 2.1.2. Spectral Approach

Fourier transform, Gabor transform, wavelet transform, and discrete cosine transform [22–24] are the representative method of spectral methods. These algorithms mentioned in this survey are listed in Table 2. Fourier transform, wavelet transform, and Gabor transform-based methods have been thoroughly studied and tested on fabric defect detection applications.Table 2
Spectral algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationSulochan [25]Multiscale wavelet features and fuzzy C-means clusteringReal and computer-simulated fabric imagesDetection error rateVermaak et al. [26]Dual-tree complex wavelet transform (DTCWT)TILDA datasetDetection success rateLiu and Zheng [27]The method based on information entropy and frequency domain saliencyDatabase created by the research associate of the industrial automation research laboratoryACC, true positive rate(TPR), false positive rate(FPR), positive predictive value (PPV), negative predictive value (NPV), time, F-measureDi et al. [28]L0 gradient minimization method and two-dimensional fractional Fourier transform (2D-FRFT) for obtaining the saliency map of the quaternion imageDataset from automation laboratory fabric database of Hong Kong UniversityTrue positive (TP), false positive (FP), true negative (TN), and false negative (FN)Jing [29]Gabor preprocessed golden image subtractionIndustrial automation laboratory at the University of Hong Kong and the TILD databaseDetection success rateMohammed and Alhamdani [30]Fuzzy back propagation neural network (FBPNN) with Gabor featuresCollected datasetDetection success rateYapi et al. [31]Using learning-based local textural distributions in the contourlet domainTILDA database(TP, FP, TN, and FN) local precision (PL), local recall (RL), and local accuracy (ACCL)Li et al. [32] propose an algorithm employing a multiscale wavelet transform and Gaussian mixture model for automated fabric defect detection. A textile fabric image was decomposed by the “Pyramid” wavelet transform and then reconstructed using thresholding method. Next, the Gaussian mixture model was utilized to segment the reconstructed image. The experiments demonstrate the effectiveness of the proposed algorithm for detecting and segmenting the defect images.Rebhi et al. [33] present a fabric defect detection approach using local homogeneity information and discrete cosine transform (DCT). DCT was applied to the newly calculated homogeneity image and different energy features of all DCT blocks are then extracted. And the extracted features are fed into the feedforward neural networks classifier.


## 2.1.3. Structural Approach

One effective way for segmenting defective area on the patterned textile fabric image is the golden image subtraction (GIS) method. Ngan et al. proposed a method named wavelet preprocessed golden image subtraction (WGIS) [34]. Additionally, wavelet transforms, GIS, and WGIS methods are compared in this study, and the proposed WGIS achieved the best performance among them.Jia and Liang [35] segment the fabric images into nonoverlapping regions named lattices and then the similarity of these lattices are calculated in the feature space. The proposed Isotropic Lattice Segmentation (ILS) method shows satisfying results on the box and star pattern image database. Jia et al. brought up another approach in their later study [36] on the basis of lattice segmentation and lattice templates. In this study, the lattices are segmented according to different placement rules of texture primitives that belong to different classes. The distances of undetermined lattice and lattice template are calculated, and the lattices are regarded as the defective area when the distances are larger than a certain threshold. The algorithm is further improved by adding template statistics which are learned from defect-free images in [37].Another template-based correction approach for fabric images with periodical structure is introduced in Chang’s work [38]. A fabric image is divided into lattices due to variation regularity and correction is then made to reduce the lattice misalignment. The defective lattices are first located and defect regions are segmented at a later step. Based on its assumptions, the lattice segmentation and template-based correction algorithms are more suitable for patterned fabric images.Shi et al. [39] propose a method using low-rank decomposition of gradient information combined with structured graph. The fabric image is first divided into defect-free regions and defect regions based on the structured graph information. Adaptive thresholding is utilized during the lattice merging step. Finally, the matrix decomposition is calculated under the prior information from the segmentation results; thus the defect regions are emphasized. The presented method outperforms other methods on a standard database.Abouelela et al. [40] design a fabric defect detection system employing simple statistical features such as median, mean, and variance. The author holds that time efficiency is crucial to any industrial procedure. Therefore, the author exchanged the complexity of simple features calculation for real-time performance. The proposed algorithm is better at detecting defects that vary drastically in the physical dimension.


## 2.1.4. Model-Based Methods

Ngan et al. propose motif-based methods for detecting defects in 2D patterned texture. This kind of methods is based on the assumption that patterned images can be divided into lattices and motifs. And further energy of moving subtraction is calculated to differentiate defective and defect-free region [41]. In order to reduce the detection rate of false positives and false negatives, the Gaussian mixture model is used to represent the energy variance value [42]. K-means clustering is applied to the data and the convex hull of each cluster is calculated in that fitting ellipsoidal region.Lucia et al. propose an algorithm for detecting the fabric defects in uniformly structured textile fabric images. The algorithm includes two stages: the feature extraction stage and the defect recognition stage. In the first stage, the symmetric Gabor filter bank and principal component analysis are utilized for feature extraction, and in the second stage, the Euclidean norm of features is calculated and compared for defect recognition. This algorithm is designed on a patch basis and proved to be effective on the public TILDA database [43].For environment-friendly textiles, Shu et al. [44] adopt an algorithm based on principal component analysis and nonlocal average filtering to enhance the fabric texture and reduce the noise. A texture-based defect measurement method is used for calculating the similarity; thus defect and nondefect areas can be distinguished.Some researchers treat the fabric defect detection problem as a one-class classification. Bu et al. [45, 46] propose a method based on the support vector data description (SVDD) model. In the training stage, the multiple fractal features are extracted and the optimal parameters are selected for the Gaussian kernel function. The detection results on several datasets demonstrate that this combination is effective. Bu et al. [47] also present another method using SVDD model, and the features extracted in this method are based on autoregressive (AR) spectral estimation model combined with Burg algorithm.With analysis on statistical model-based methods, Campbell et al. [48] propose two model-based methods in their study. The first method states the maximum likelihood of image binarization. The other method is mainly for defect detection in repeated weaving patterns; thus the discrete Fourier transformation is utilized for texture analysis. In the end, a model-based clustering method is applied to delineate the defective regions.For the detection of patterned fabrics, Tsang et al. [49] propose a method named Elo rating (ER) in line with the spirit of sports. The fabric images are divided into standard-size partitions. Matches are calculated between partitions and revised through an Elo point matrix. The defect area (partition) will win the competition as a powerful player. This presented method was tested on dot-, star-, and box-patterned fabrics database and obtained comparable results to the most advanced method.


## 2.2. Learning-Based Algorithms

### 2.2.1. Classical Machine Learning Algorithms

(1) Dictionary Learning-Based Algorithms. Many researchers have validated the effectiveness of dictionary learning-based algorithms dealing with textile fabric defect detection problems [50, 51]. The general steps of these algorithms: first a dictionary is learned from the training or test image, and then a fabric image without defects is reconstructed using the learned dictionary; thenceforth the detection is implemented by subtracting the reconstructed image from the test image. Recently, many algorithms based on low-rank representation have been brought up in the application of fabric defect inspection. In order to solve the optimization problem of the objective function, many methods reduce the low-rank decomposition problem to the nuclear minimization (NNM) problem.Li et al. propose an algorithm on the basis of biological vision modeling. The biological visual saliency is modeled by low-rank representation (LRR); thus the fabric image is decomposed into salient defect regions and defect-free backgrounds [52].Li et al. [53] model a defect-free region as a low-rank structure and the defect region as a sparse structure. Thus a fabric image can be regarded as the sum of a low-rank matrix and a sparse matrix. For dimensionality reduction, instead of singular value decomposition (SVD) on the matrix of the original image, the presented method uses eigenvalue decomposition on blocked image matrix. Therefore, this method is easy to implement and works well given that the contrast of the fabric image is sufficient.Table3tabulates some other dictionary learning-based algorithms using low-rank decomposition.Table 3
Dictionary learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationLi et al. [52]Low-rank representation (LRR)(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryPrecision and recallLi et al. [53]Low-rank representation500 fabric images from the textile kind C1 of the TILDA database(a) Sensitivity and specificity; (b) false alarm rate (FAR), missing rate (MR)Gao et al. [54]Gabor filter and tensor low-rank recoveryDataset from the research associate of industrial automation research laboratoryReceiver operating characteristic curve (ROC)Shi et al. [39, 47]Low-rank decomposition with gradient informationDataset from the research associate of industrial automation research laboratoryTPR, FPR, PPV, NPVLiu et al. [55–57]Multi-scale convolutional neural network and low-rank decomposition model(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryMeans and standard deviations of average precisions, recalls, F-measure, and mean absolute error (MAE)Mo et al. [58]Weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weightsDatabase is from the research associate of industrial automation research laboratory, HKBUVisual defect locating results, the metrics of false alarm, recall, precision, accuracy, and F-measureLi et al. [59]Low-rank decomposition of multichannel feature matrices(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryROC curves and precision-recall (PR) curvesYang et al. [60]Sparse and dense mixed low-rank decompositionReal-world samples of 512∗512 with 256-gray levelsSaliency map (qualitative)Wang et al. [61]A randomized low-rank and sparse matrix decomposition model named GoDecFabric image dataset collected by Dr. Henry Y. T. Ngan [62]Precision, recall, and F-measureShi et al. [39] point out two shortcomings of the low-rank decomposition. One is that existing low-rank decomposition models barely detect the defect regions with high gradients. And the other shortcoming is that small defect area or complex area will be incorrectly segmented given the inaccuracy of prior information. To overcome these shortcomings, Shi et al. propose a low-rank decomposition method utilizing gradient information combined with structured graph algorithm. The proposed method outperforms others on the point, box, and star databases.Traditional machine learning algorithms, such as KNN [63] and neural network [64], are widely used in fabric defection detection problems. And feature engineering is one of the major processes in the machine learning life cycle.Mak et al. extracted [65] four novel fractal features and employed support vector data description (SVDD), which is a support vector machine learning algorithm used for one-class classification, for fabric defect detection in his work.Zhang et al. propose an approach employing the radial basis function (RBF) network. Gaussian mixture model (GMM) is utilized to improve the accuracy of Gaussian RBF parameter estimation. The validity of the proposed method has been proved on multiple class datasets [66].Tian and Li [67] propose an autoencoder-based method for fabric defect detection by exploring similarities between image patches. Utilizing the repeated texture pattern, similar nondefective patches were found for each candidate defect patch and the corresponding latent variables were weighted and combined according to which the original latent variable can be modified. Experimental results manifest the effectiveness of the proposed algorithm.Yapi et al. [68] consider this problem as a binary problem. A compact and accurate feature set was extracted by statistical modeling of multiscale contourlet decomposition, and then a Bayesian classifier (BC) is used to classify the defect and nondefect classes. This algorithm obtained high precision detection with real-time efficiency.Some other traditional machine learning algorithms for fabric defect detection are shown in Table4.Table 4
Traditional machine learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationWang et al. [63]Multiview stereo vision (MVS) and bag-of-features (BOF), K-nearest neighbor (KNN) algorithmCollected datasetDetection success ratePriyanka and Manish [69]Artificial neural networks (ANN)Collected datasetDetection success rateBumrungkun [70]Snake active contour and support vector machinesCollected datasetRecognition accuracy detection success rateZhang et al. [71]L0 gradient minimization (LGM) and the fuzzy c-means (FCM) method to detect various fabric defects with diverse texturesImages from the automation laboratory sample database of Hong Kong University, TILDA textile texture database, and Guang Dong Esquel TextilesACC, TPR, FPR, PPV, and IOU (intersection over union)


### 2.2.2. Deep Learning Algorithms

Recently, many researchers have applied deep learning techniques to fabric defect detection problems and have achieved satisfying results [72, 73] for the improvement of textile product quality and production efficiency [74]. Although deep learning methods have been proved to be powerful when dealing with segmentation and classification problems, there are still some problems in the practical application of specific industries [75]. First of all, the actual textile production line requires high real-time performance of the algorithm, which is the demand of high execution efficiency. Furthermore, compared to normal defect-free samples, the defective image data is difficult to obtain, which brings challenges to the training process of deep learning [76].At present, the deep learning-based object detector can be classified as one-stage detectors and two-stage detectors [77]. Classical deep learning algorithms for object detection are listed in Table 5. In general, one-stage detectors have fast detection speed to meet the requirements of online detection, but the detection accuracy usually fails to meet requirements. In contrast, the two-stage algorithms have higher detection accuracy, but its detection speed is difficult to meet the real-time requirements of the algorithm in production scenes. In the field of fabric defect detection, the advantages and disadvantages of one-stage and two-stage detection algorithms are quite similar to those in other fields. The two-stage algorithm has higher accuracy but slower speed than the one-stage algorithm. In the actual application of textile industry, we hope that, under the premise of satisfying the detection accuracy, the faster the detection speed, the better. Therefore, the algorithm should be selected according to the actual application scenarios and requirements to find the balance between efficiency and accuracy.Table 5
Deep learning algorithms for detecting object.
One-stage detectorsTwo-stage detectorsYOLOFaster RCNNSSDMask RCNNYOLOv2/v3/v4Cascade RCNNRefineDetFPNRetinaNetR-FCN(1) One-Stage Detection Algorithms. One-stage detection algorithm does not have a separate proposal generation phase. Typically, these algorithms treat all locations on the image as potential objects and manage to categorize each interest region into a target object or background.The recently proposed single shot multibox detector (SSD) is a typical one-stage detector that has obtained good detection performance in object detection. This algorithm is designed based on a convolutional neural network (CNN). Some improvements have been made for the fabrics defect scenario by Liu et al. [78] and the experimental results show rationality and effectiveness.Ouyang et al. [79] present a CNN based algorithm for on-loom fabric defect inspection. This proposed algorithm introduces a dynamic activation layer utilizing the defect probability information with a pairwise potential function to a CNN. This algorithm obtains good results dealing with the unbalanced data classification problem.Deep convolutional neural network (DCNN) based algorithms have achieved satisfactory results on visual tasks and have been widely used in industrial scenarios. Liu et al. [80] employ DCNN to detect fabric defects with complicated textures. This proposed approach is particularly designed for real textile production environment with limited resources. A series of improvements have been done to make the detection more effective. Zhou et al. propose an efficient DCNN architecture focusing on the problem of fabric defect detection, called Efficient Defect Detectors (EDDs) [81]. To extract more low-level features, EDDs adjust the input resolution, depth, and width using a scaling strategy. The improvement proved to be effective when compared with existing fabric defect detection algorithms.Xu et al. [82] propose a novel detection network named de-deformation defect detection network (D4Net). This model is composed of reference generation, de-deformation network, and marginal loss. The most suitable reference is selected and paired with the input image and then is sent into the de-deformation network. The dissimilarities are calculated and enhanced by the marginal loss. Experiments on a large industrial database containing 67K images have been done and the results show that the algorithm outperforms other algorithms especially for large pattern fabric images.Peng et al. put forward a detection algorithm called Priori Anchor Convolutional Neural Network (PRAN-Net) to fix this problem. Feature Pyramid Network (FPN) is utilized to selected multiscale feature maps and then sparse priori anchors are generated based on ground truth boxes [83].Li et al. [84] propose an architecture using several microarchitectures. The microarchitecture is constructed of multiscale analysis, filter factorization, multilocation pooling, and parameters reduction, thus making the network a compact one. With the small model size, the proposed net worked well on fabric defect detection.(2) Two-Stage Detection Algorithms. With regard to two-stage detectors, a sparse set of proposals is generated in the first stage and in the second stage, and the features of generated proposals are sent into DCNN for prediction results. As a remarkable two-stage detector, Faster R-CNN is an object detection model that improves on Fast R-CNN by utilizing a region proposal network (RPN) with the CNN model. Some researchers utilize the modified Faster R-CNN model for fabric defect detection [85–87]. Jun et al. [88] propose a framework that utilized the Inception-V1 model and LeNet-5 model. This approach includes local defect prediction in the first stage and global defect recognition in the second stage.Table6 lists some other deep learning algorithms utilized in textile fabric defect applications.Table 6
Deep learning-based algorithms for fabric defect detection.
AuthorProposed or tested modelCategorizeDatasetEvaluationJing et al. [89]Mobile-UnetOne-stageBenchmark databases, the fabric images database (FID), and yarn dyed fabric images (YFI), in which all images are manually annotated segmentationPixel accuracy (PA) and IoUHong-wei hang [90]YOLOV2One-stageCollected dataset (276)IOU, recall, and precisionYoung-Joo Han [91]Stacked convolutional autoencodersOne-stageSynthetic and collected datasetRecall, precision, and F-scoreXinying He [92]Adaptive method based on DenseNet-SSDOne-stageCollected dataset (2072)Calculate localization loss (loc) and confidence loss (conf)Mohammed et al. [93]A multilayer perceptron with a Levenberg–Marquardt (LM) algorithmOne-stageCollected dataset (217)Specificity, accuracy, and sensitivityShuang mei [94]Multiscale convolutional denoising autoencoder network modelOne-stageFour datasets: fabrics, KTH-TIPS, Kylberg texture, and ms-textureRecall, precision, and F1-measureHuosheng Xie [95]Improved RefineDetOne-stageTILDA dataset, Hong Kong patterned textures database, and DAGM2007 datasetPrecision (P), recall I, F1-score, mean average precision (mAP), model parameter (param.)Yanqing Huang [96]Segmentation network and decision networkTwo-stageDark redfFabric (DRF), light blue fabric (LBF) and patterned texture fabric (PTF)Frames per second (FPS) Avg-IoU and Avg-PIn recent years, Generative Adversarial Networks (GANs) have attracted a lot of attention [97, 98]. GANs and related algorithms have been widely used in a range of computer vision and computer graphics applications, such as image synthesis and video generation. GAN based fabric defect detection algorithms can automatically adapt to different fabric textures by learning existing fabric defect samples [99]. Liu et al. design a deep semantic segmentation network to detect fabric defects. They train a multistage GAN model to synthesize reasonable defect samples from nondefect samples. The performance of the method was verified by comprehensive experiments on all sorts of typical fabric image samples.Le et al. [100] utilize Wasserstein generative adversarial nets (WGANs) combined with transfer learning techniques and multimodel ensembling framework. The effectiveness of the proposed scheme is demonstrated on unbalanced and rare datasets of images with defects.Inspired by biological visual perception mechanism, Zhao et al. [101] describe a CNN model based on visual long-short-term memory (VLSTM). Three types of features, visual perception features, visual short-term memory (VSTM) features, and visual long-term memory (VLTM) features, are extracted by stacked convolutional autoencoders, a shallow CNN, and nonlocal neural networks, respectively. Experiments have been done on three public datasets and results show that the proposed algorithm is comparable to state-of-the-art algorithms.Traditional saliency detection models usually rely on hand-crafted features to capture the information of global context and local details. Researchers add the attention mechanism to deal with fabric defect detection problems. Wang et al. [102] propose a deep saliency detection model that incorporated self-attention mechanism into a CNN for fabric defect detection. Multiscale feature maps are generated from a fully convolutional network, and a self-attention module is used to coordinate the dependence between the features of different layers. The self-attention mechanism in this algorithm proved to be very effective with complex or blurred defects.Some studies combine the traditional and deep learning methods. Wang et al. [103] extracted global deep features using CNN in combination with handcrafted low-level features, and nonconvex robust PCA regularized by nonconvex total variation are employed to data processing and noise reduction. Then a segmentation algorithm is used to segment the saliency map to obtain the defect area.


## 2.2.1. Classical Machine Learning Algorithms

(1) Dictionary Learning-Based Algorithms. Many researchers have validated the effectiveness of dictionary learning-based algorithms dealing with textile fabric defect detection problems [50, 51]. The general steps of these algorithms: first a dictionary is learned from the training or test image, and then a fabric image without defects is reconstructed using the learned dictionary; thenceforth the detection is implemented by subtracting the reconstructed image from the test image. Recently, many algorithms based on low-rank representation have been brought up in the application of fabric defect inspection. In order to solve the optimization problem of the objective function, many methods reduce the low-rank decomposition problem to the nuclear minimization (NNM) problem.Li et al. propose an algorithm on the basis of biological vision modeling. The biological visual saliency is modeled by low-rank representation (LRR); thus the fabric image is decomposed into salient defect regions and defect-free backgrounds [52].Li et al. [53] model a defect-free region as a low-rank structure and the defect region as a sparse structure. Thus a fabric image can be regarded as the sum of a low-rank matrix and a sparse matrix. For dimensionality reduction, instead of singular value decomposition (SVD) on the matrix of the original image, the presented method uses eigenvalue decomposition on blocked image matrix. Therefore, this method is easy to implement and works well given that the contrast of the fabric image is sufficient.Table3tabulates some other dictionary learning-based algorithms using low-rank decomposition.Table 3
Dictionary learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationLi et al. [52]Low-rank representation (LRR)(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryPrecision and recallLi et al. [53]Low-rank representation500 fabric images from the textile kind C1 of the TILDA database(a) Sensitivity and specificity; (b) false alarm rate (FAR), missing rate (MR)Gao et al. [54]Gabor filter and tensor low-rank recoveryDataset from the research associate of industrial automation research laboratoryReceiver operating characteristic curve (ROC)Shi et al. [39, 47]Low-rank decomposition with gradient informationDataset from the research associate of industrial automation research laboratoryTPR, FPR, PPV, NPVLiu et al. [55–57]Multi-scale convolutional neural network and low-rank decomposition model(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryMeans and standard deviations of average precisions, recalls, F-measure, and mean absolute error (MAE)Mo et al. [58]Weighted double-low-rank decomposition method (WDLRD) to treat the matrix singular values differently by assigning different weightsDatabase is from the research associate of industrial automation research laboratory, HKBUVisual defect locating results, the metrics of false alarm, recall, precision, accuracy, and F-measureLi et al. [59]Low-rank decomposition of multichannel feature matrices(1) TILDA fabric images dataset; (2) dataset from the research associate of industrial automation research laboratoryROC curves and precision-recall (PR) curvesYang et al. [60]Sparse and dense mixed low-rank decompositionReal-world samples of 512∗512 with 256-gray levelsSaliency map (qualitative)Wang et al. [61]A randomized low-rank and sparse matrix decomposition model named GoDecFabric image dataset collected by Dr. Henry Y. T. Ngan [62]Precision, recall, and F-measureShi et al. [39] point out two shortcomings of the low-rank decomposition. One is that existing low-rank decomposition models barely detect the defect regions with high gradients. And the other shortcoming is that small defect area or complex area will be incorrectly segmented given the inaccuracy of prior information. To overcome these shortcomings, Shi et al. propose a low-rank decomposition method utilizing gradient information combined with structured graph algorithm. The proposed method outperforms others on the point, box, and star databases.Traditional machine learning algorithms, such as KNN [63] and neural network [64], are widely used in fabric defection detection problems. And feature engineering is one of the major processes in the machine learning life cycle.Mak et al. extracted [65] four novel fractal features and employed support vector data description (SVDD), which is a support vector machine learning algorithm used for one-class classification, for fabric defect detection in his work.Zhang et al. propose an approach employing the radial basis function (RBF) network. Gaussian mixture model (GMM) is utilized to improve the accuracy of Gaussian RBF parameter estimation. The validity of the proposed method has been proved on multiple class datasets [66].Tian and Li [67] propose an autoencoder-based method for fabric defect detection by exploring similarities between image patches. Utilizing the repeated texture pattern, similar nondefective patches were found for each candidate defect patch and the corresponding latent variables were weighted and combined according to which the original latent variable can be modified. Experimental results manifest the effectiveness of the proposed algorithm.Yapi et al. [68] consider this problem as a binary problem. A compact and accurate feature set was extracted by statistical modeling of multiscale contourlet decomposition, and then a Bayesian classifier (BC) is used to classify the defect and nondefect classes. This algorithm obtained high precision detection with real-time efficiency.Some other traditional machine learning algorithms for fabric defect detection are shown in Table4.Table 4
Traditional machine learning algorithms for fabric defect detection.
AuthorProposed methodDatasetEvaluationWang et al. [63]Multiview stereo vision (MVS) and bag-of-features (BOF), K-nearest neighbor (KNN) algorithmCollected datasetDetection success ratePriyanka and Manish [69]Artificial neural networks (ANN)Collected datasetDetection success rateBumrungkun [70]Snake active contour and support vector machinesCollected datasetRecognition accuracy detection success rateZhang et al. [71]L0 gradient minimization (LGM) and the fuzzy c-means (FCM) method to detect various fabric defects with diverse texturesImages from the automation laboratory sample database of Hong Kong University, TILDA textile texture database, and Guang Dong Esquel TextilesACC, TPR, FPR, PPV, and IOU (intersection over union)


## 2.2.2. Deep Learning Algorithms

Recently, many researchers have applied deep learning techniques to fabric defect detection problems and have achieved satisfying results [72, 73] for the improvement of textile product quality and production efficiency [74]. Although deep learning methods have been proved to be powerful when dealing with segmentation and classification problems, there are still some problems in the practical application of specific industries [75]. First of all, the actual textile production line requires high real-time performance of the algorithm, which is the demand of high execution efficiency. Furthermore, compared to normal defect-free samples, the defective image data is difficult to obtain, which brings challenges to the training process of deep learning [76].At present, the deep learning-based object detector can be classified as one-stage detectors and two-stage detectors [77]. Classical deep learning algorithms for object detection are listed in Table 5. In general, one-stage detectors have fast detection speed to meet the requirements of online detection, but the detection accuracy usually fails to meet requirements. In contrast, the two-stage algorithms have higher detection accuracy, but its detection speed is difficult to meet the real-time requirements of the algorithm in production scenes. In the field of fabric defect detection, the advantages and disadvantages of one-stage and two-stage detection algorithms are quite similar to those in other fields. The two-stage algorithm has higher accuracy but slower speed than the one-stage algorithm. In the actual application of textile industry, we hope that, under the premise of satisfying the detection accuracy, the faster the detection speed, the better. Therefore, the algorithm should be selected according to the actual application scenarios and requirements to find the balance between efficiency and accuracy.Table 5
Deep learning algorithms for detecting object.
One-stage detectorsTwo-stage detectorsYOLOFaster RCNNSSDMask RCNNYOLOv2/v3/v4Cascade RCNNRefineDetFPNRetinaNetR-FCN(1) One-Stage Detection Algorithms. One-stage detection algorithm does not have a separate proposal generation phase. Typically, these algorithms treat all locations on the image as potential objects and manage to categorize each interest region into a target object or background.The recently proposed single shot multibox detector (SSD) is a typical one-stage detector that has obtained good detection performance in object detection. This algorithm is designed based on a convolutional neural network (CNN). Some improvements have been made for the fabrics defect scenario by Liu et al. [78] and the experimental results show rationality and effectiveness.Ouyang et al. [79] present a CNN based algorithm for on-loom fabric defect inspection. This proposed algorithm introduces a dynamic activation layer utilizing the defect probability information with a pairwise potential function to a CNN. This algorithm obtains good results dealing with the unbalanced data classification problem.Deep convolutional neural network (DCNN) based algorithms have achieved satisfactory results on visual tasks and have been widely used in industrial scenarios. Liu et al. [80] employ DCNN to detect fabric defects with complicated textures. This proposed approach is particularly designed for real textile production environment with limited resources. A series of improvements have been done to make the detection more effective. Zhou et al. propose an efficient DCNN architecture focusing on the problem of fabric defect detection, called Efficient Defect Detectors (EDDs) [81]. To extract more low-level features, EDDs adjust the input resolution, depth, and width using a scaling strategy. The improvement proved to be effective when compared with existing fabric defect detection algorithms.Xu et al. [82] propose a novel detection network named de-deformation defect detection network (D4Net). This model is composed of reference generation, de-deformation network, and marginal loss. The most suitable reference is selected and paired with the input image and then is sent into the de-deformation network. The dissimilarities are calculated and enhanced by the marginal loss. Experiments on a large industrial database containing 67K images have been done and the results show that the algorithm outperforms other algorithms especially for large pattern fabric images.Peng et al. put forward a detection algorithm called Priori Anchor Convolutional Neural Network (PRAN-Net) to fix this problem. Feature Pyramid Network (FPN) is utilized to selected multiscale feature maps and then sparse priori anchors are generated based on ground truth boxes [83].Li et al. [84] propose an architecture using several microarchitectures. The microarchitecture is constructed of multiscale analysis, filter factorization, multilocation pooling, and parameters reduction, thus making the network a compact one. With the small model size, the proposed net worked well on fabric defect detection.(2) Two-Stage Detection Algorithms. With regard to two-stage detectors, a sparse set of proposals is generated in the first stage and in the second stage, and the features of generated proposals are sent into DCNN for prediction results. As a remarkable two-stage detector, Faster R-CNN is an object detection model that improves on Fast R-CNN by utilizing a region proposal network (RPN) with the CNN model. Some researchers utilize the modified Faster R-CNN model for fabric defect detection [85–87]. Jun et al. [88] propose a framework that utilized the Inception-V1 model and LeNet-5 model. This approach includes local defect prediction in the first stage and global defect recognition in the second stage.Table6 lists some other deep learning algorithms utilized in textile fabric defect applications.Table 6
Deep learning-based algorithms for fabric defect detection.
AuthorProposed or tested modelCategorizeDatasetEvaluationJing et al. [89]Mobile-UnetOne-stageBenchmark databases, the fabric images database (FID), and yarn dyed fabric images (YFI), in which all images are manually annotated segmentationPixel accuracy (PA) and IoUHong-wei hang [90]YOLOV2One-stageCollected dataset (276)IOU, recall, and precisionYoung-Joo Han [91]Stacked convolutional autoencodersOne-stageSynthetic and collected datasetRecall, precision, and F-scoreXinying He [92]Adaptive method based on DenseNet-SSDOne-stageCollected dataset (2072)Calculate localization loss (loc) and confidence loss (conf)Mohammed et al. [93]A multilayer perceptron with a Levenberg–Marquardt (LM) algorithmOne-stageCollected dataset (217)Specificity, accuracy, and sensitivityShuang mei [94]Multiscale convolutional denoising autoencoder network modelOne-stageFour datasets: fabrics, KTH-TIPS, Kylberg texture, and ms-textureRecall, precision, and F1-measureHuosheng Xie [95]Improved RefineDetOne-stageTILDA dataset, Hong Kong patterned textures database, and DAGM2007 datasetPrecision (P), recall I, F1-score, mean average precision (mAP), model parameter (param.)Yanqing Huang [96]Segmentation network and decision networkTwo-stageDark redfFabric (DRF), light blue fabric (LBF) and patterned texture fabric (PTF)Frames per second (FPS) Avg-IoU and Avg-PIn recent years, Generative Adversarial Networks (GANs) have attracted a lot of attention [97, 98]. GANs and related algorithms have been widely used in a range of computer vision and computer graphics applications, such as image synthesis and video generation. GAN based fabric defect detection algorithms can automatically adapt to different fabric textures by learning existing fabric defect samples [99]. Liu et al. design a deep semantic segmentation network to detect fabric defects. They train a multistage GAN model to synthesize reasonable defect samples from nondefect samples. The performance of the method was verified by comprehensive experiments on all sorts of typical fabric image samples.Le et al. [100] utilize Wasserstein generative adversarial nets (WGANs) combined with transfer learning techniques and multimodel ensembling framework. The effectiveness of the proposed scheme is demonstrated on unbalanced and rare datasets of images with defects.Inspired by biological visual perception mechanism, Zhao et al. [101] describe a CNN model based on visual long-short-term memory (VLSTM). Three types of features, visual perception features, visual short-term memory (VSTM) features, and visual long-term memory (VLTM) features, are extracted by stacked convolutional autoencoders, a shallow CNN, and nonlocal neural networks, respectively. Experiments have been done on three public datasets and results show that the proposed algorithm is comparable to state-of-the-art algorithms.Traditional saliency detection models usually rely on hand-crafted features to capture the information of global context and local details. Researchers add the attention mechanism to deal with fabric defect detection problems. Wang et al. [102] propose a deep saliency detection model that incorporated self-attention mechanism into a CNN for fabric defect detection. Multiscale feature maps are generated from a fully convolutional network, and a self-attention module is used to coordinate the dependence between the features of different layers. The self-attention mechanism in this algorithm proved to be very effective with complex or blurred defects.Some studies combine the traditional and deep learning methods. Wang et al. [103] extracted global deep features using CNN in combination with handcrafted low-level features, and nonconvex robust PCA regularized by nonconvex total variation are employed to data processing and noise reduction. Then a segmentation algorithm is used to segment the saliency map to obtain the defect area.


## 3. Application and Deployment

When it comes to the deployment phase, there will be a lot of engineering implementation problems [104, 105]. Realizing an intelligent textile system in the real textile manufacturing process covers many aspects, involving the Internet of Things (IoT) [106], cyber-physical systems (CPS) [107], and more [108, 109].


### 3.1. Hardware Selection of Detection System

The basic components of image acquisition system are very important for the deployment work and therefore the hardware selection is critical for subsequent detection work [110]. Hardware such as cameras, lens, lights, and frame grabber is an important factor. Different hardware corresponds to different subsequent algorithms. For instance, Yildiz et al. [111] present a new fabric defect detection method for using a thermal camera. Fabric images obtained by the thermal imaging camera have their own image characteristics. The algorithm can be designed utilizing thermal differences between defect and defect-free areas and to improve the detection accuracy and efficiency while reducing the cost.Different from visual inspection systems, Fang et al. [112] introduce a tactile inspection system for fabric defect detection. The system design is mainly based on a visual tactile sensor, which consists of several LEDs, a camera, and an elastic sensing layer. This system captures detailed information of surface structure neglecting of color and pattern; thus the algorithm designed in this study is mainly based on the structural information obtained from the tactile sensor.In addition, the conveyor belt used for conveying cloth on the production line will also affect the image taking speed [113]. Therefore, the hardware selection of the entire system needs to be considered as a whole.


### 3.2. Dataset

This review lists a lot of learning-based algorithms. Although this type of method is very effective, it requires a large number of labeled fabric images with defects. However, it is very difficult to collect a fair amount of fabric defect image data in industrial scenes [114]. Therefore, many researchers employ semisupervised and unsupervised learning algorithms for the detection [115]. In addition, some studies utilize nondefect image data and synthetic defective image data generated by using defect characteristics based on expert knowledge [91]. Chen et al. [116] propose a data augmentation method based on automatic image acquisition. Different image acquisition angles, various acquisition scenes, and random illumination conditions are designed for image collection as a simulation under the actual textile production scene.In addition, the diversity of fabric defects and unbalanced categories bring actual challenges to fabric defect detection [86]. For instance, the multiscale defects in the fabric image will greatly increase the complexity and computation of the model; thus the designed fabric defect detection model of appropriate size must be able to meet the multiscale target detection.


### 3.3. Real Time of the Algorithm

Generally, the actual fabric defect detection task is implemented online on a platform with limited computing power. Thus the algorithms of online defect detection systems need to be accurate, efficient, and robust [117]. Therefore, the robustness and efficiency of the algorithm are critical to the actual production line. The computational cost of different algorithms is a critical consideration.However, there are many other problems that exist in the textile production scenario. In order to obtain better image data and detection results, most existing algorithms require textile to be flattened. Therefore, some scholars design defect detection algorithms specifically for textiles with uneven and diverse shapes [118]. This consideration is closer to real-world settings.Shunji et al. [119] design a detection method for tubular knitted fabric which is produced by a circular knitting machine. Vertical defects in circular knitted fabrics are caused by damaged needles. Once a vertical defect occurs during knitting, it will continue to exist unless the damaged needle is replaced with a new one.In general, there is no real-time quality control system that can guarantee the quality in the production of noncrimp fabrics. The embedded system proposed by Schmitt et al. [120] ensures that all steps of image acquisition, processing, and evaluation can be executed in real time. The advantage of this proposed system is that real-time, accurate, and robust performance of the algorithm is ensured by detecting fiber orientation under industrial conditions.In practical applications, fabric defect detection algorithms must not only ensure detection accuracy but also guarantee its applicability to hardware platforms with limited resources. Currently, the accuracy of existing detection models is low. This is due in large part to multiscale defects in the fabric image; so, the fabric defect detection model must be able to meet multiscale object detection. However, even the best model is still troubled by the large size of the problem. Therefore, we must consider ways to reduce the size of the model. Inspired by the successful use of deep convolutional neural networks (DCNN) for target detection, we propose a wide-and-light network structure called WALNet.


## 3.1. Hardware Selection of Detection System

The basic components of image acquisition system are very important for the deployment work and therefore the hardware selection is critical for subsequent detection work [110]. Hardware such as cameras, lens, lights, and frame grabber is an important factor. Different hardware corresponds to different subsequent algorithms. For instance, Yildiz et al. [111] present a new fabric defect detection method for using a thermal camera. Fabric images obtained by the thermal imaging camera have their own image characteristics. The algorithm can be designed utilizing thermal differences between defect and defect-free areas and to improve the detection accuracy and efficiency while reducing the cost.Different from visual inspection systems, Fang et al. [112] introduce a tactile inspection system for fabric defect detection. The system design is mainly based on a visual tactile sensor, which consists of several LEDs, a camera, and an elastic sensing layer. This system captures detailed information of surface structure neglecting of color and pattern; thus the algorithm designed in this study is mainly based on the structural information obtained from the tactile sensor.In addition, the conveyor belt used for conveying cloth on the production line will also affect the image taking speed [113]. Therefore, the hardware selection of the entire system needs to be considered as a whole.


## 3.2. Dataset

This review lists a lot of learning-based algorithms. Although this type of method is very effective, it requires a large number of labeled fabric images with defects. However, it is very difficult to collect a fair amount of fabric defect image data in industrial scenes [114]. Therefore, many researchers employ semisupervised and unsupervised learning algorithms for the detection [115]. In addition, some studies utilize nondefect image data and synthetic defective image data generated by using defect characteristics based on expert knowledge [91]. Chen et al. [116] propose a data augmentation method based on automatic image acquisition. Different image acquisition angles, various acquisition scenes, and random illumination conditions are designed for image collection as a simulation under the actual textile production scene.In addition, the diversity of fabric defects and unbalanced categories bring actual challenges to fabric defect detection [86]. For instance, the multiscale defects in the fabric image will greatly increase the complexity and computation of the model; thus the designed fabric defect detection model of appropriate size must be able to meet the multiscale target detection.


## 3.3. Real Time of the Algorithm

Generally, the actual fabric defect detection task is implemented online on a platform with limited computing power. Thus the algorithms of online defect detection systems need to be accurate, efficient, and robust [117]. Therefore, the robustness and efficiency of the algorithm are critical to the actual production line. The computational cost of different algorithms is a critical consideration.However, there are many other problems that exist in the textile production scenario. In order to obtain better image data and detection results, most existing algorithms require textile to be flattened. Therefore, some scholars design defect detection algorithms specifically for textiles with uneven and diverse shapes [118]. This consideration is closer to real-world settings.Shunji et al. [119] design a detection method for tubular knitted fabric which is produced by a circular knitting machine. Vertical defects in circular knitted fabrics are caused by damaged needles. Once a vertical defect occurs during knitting, it will continue to exist unless the damaged needle is replaced with a new one.In general, there is no real-time quality control system that can guarantee the quality in the production of noncrimp fabrics. The embedded system proposed by Schmitt et al. [120] ensures that all steps of image acquisition, processing, and evaluation can be executed in real time. The advantage of this proposed system is that real-time, accurate, and robust performance of the algorithm is ensured by detecting fiber orientation under industrial conditions.In practical applications, fabric defect detection algorithms must not only ensure detection accuracy but also guarantee its applicability to hardware platforms with limited resources. Currently, the accuracy of existing detection models is low. This is due in large part to multiscale defects in the fabric image; so, the fabric defect detection model must be able to meet multiscale object detection. However, even the best model is still troubled by the large size of the problem. Therefore, we must consider ways to reduce the size of the model. Inspired by the successful use of deep convolutional neural networks (DCNN) for target detection, we propose a wide-and-light network structure called WALNet.


## 4. Discussion

Fabric defects correspond to defects on the surface of the textile fabric. Most fabric defects are caused by machine or process faults and malfunctions. The existence of fabric defects greatly reduces the sale and use of textiles. Textile manufacturing companies need to upgrade equipment and technology to maintain growth and competitiveness.The sensing, storage, and computing capabilities of automated fabric detection systems based on computer vision will continue to improve. The development of hardware and algorithms will greatly affect the accuracy of detection and the ease of deployment.Besides the fabric defect detection phase discussed in this survey, there is a lot of work that needs to be done during the whole textile manufacturing process. A lot of research works have been proposed for yarn production, fabric manufacturing, and finishing process utilizing learning-based methods. Huynh [121] proposes an online fabric defect prediction method based on the back propagation neural network models. The acquired data is collected in the form of time series and then converted into regional data based on the control chart. The proposed model can predict the defect types in advance and thus can reduce the workload of quality control in the production process.In the future, more work needs to be done in the process of moving towards Industry 4.0 [122, 123]. Smart manufacturing integrates various technologies, covering robotics, CPS, IoT, big data analytics [124], and cloud computing. CPS is an engineering system that seamlessly integrates physical and computational components [125]. Adding artificial intelligence, big data analysis, and cloud services to the IoT ecosystem is the key development direction of CPS in the future.


## 5. Conclusions

This paper presents a systematic literature review on automatic fabric defect detection methods of the textile industry smart manufacturing. All the methods covered in this work are roughly classified into two main categories, namely, traditional algorithms and learning-based algorithms. There are no clear boundaries between the different categories. To realize a better detection result, the researchers often combine different algorithms. The research results of this survey also confirmed that better results can be obtained by combining different methods and thus provide suggestions and ideas for further research. Accurate, efficient, and robust fabric defect detection algorithms are necessary to develop fully automated web detection systems.The automatic textile fabric defect detection technology based on computer vision has attracted great attention of researchers. With the development of new object detection algorithms, computational capabilities, and sensor technology and industry, computer-vision based textile defect detection techniques will continue to evolve at a high speed.



---

*Source: 9948808-2021-05-10.xml*