## 2 Related Work  
### Deep Learning-Based Defect Detection Methods  
In the literature, there are CNN-based, transformer-based, and hybrid models where CNN and transformers are integrated into deep learning-based defect detection studies. These studies aim to improve the performance of surface defect detection with different architectural approaches and algorithms. Methods based on both traditional CNN architectures and next-generation approaches have been developed.  
A high-accuracy Faster R-CNN algorithm is proposed for fabric surface defect detection, developed for a custom-prepared dataset [35]. For identifying surface defects on railway, a novel network based on Mask R-CNN was developed, achieving an average accuracy of 98.70% [36]. The approach of RAF-SSD combined the attention mechanism along with a multi-feature fusion network and a 12.9% increase in detection rate compared to the traditional deep learning model proposed by [PERSON] and [PERSON] [37].  
Transformers, initially popularised in natural language processing, are increasingly gaining attention in computer vision and deep learning-based defect detection due to their key advantages: the attention mechanism, long-range dependency capture, and transfer learning capabilities.  
Unlike CNNs operating directly on image pixels, transformers divide an input image into patches of fixed pixel size [38]. Then, they propose a new network architecture, the LSWin Transformer, for the detection of steel surface defects, which showed very promising performance in surface defect detection [39; 40]. Another study, named ETDNet, on steel surface defects effectively classified defects of various shapes and sizes [41]. [PERSON] et al. [42] proposed an intelligent surface defect detection method called DATNet, employing a Transformer-based encoder to prevent blade failure and tool wear.  
Transformers are general-purpose models that can be adapted for both NLP and computer vision tasks. They have an attentional mechanism that models long-distance dependencies between image regions to dynamically focus on different parts, which makes them effective in variable-sized images or images with irregular structures [43]. However, the self-attention mechanism requires a large computational cost, thus limiting its use on resource-constrained devices, and requires a large dataset for performance reasons. Training and fine-tuning processes, especially the identification of hyperparameters, can be quite challenging [44; 45].  
CNNs can be successful in extracting low-level features. The natural modelling of locality and translation invariance by CNN fits into image processing [46], making it a great choice for resource-constrained or real-time devices [47]. Quite often, with less data, it is possible to come up with more plausible good performance [48]. In the meantime, local connections are always the primary focus in CNNs, and, to catch dependencies in larger scales, extra layers or more complex architecture are often needed [49]. These two important architectures, when embedded in deep learning defect detection, can revolutionise quality control in an industrial process. Any architecture is thus completely dependent on application and task, since every model may prove most appropriate regarding the considered factors.