## 3 The Framework of the Proposed Method and the Data Required for the Experiment
### Overall Framework
#### 3.1.1 Image Classification in Computer Vision
As shown in Figure 3b-d, each element of the convolution kernel is shifted according to certain rules and the sampling position is also shifted, which improves the recognition and detection ability of irregular objects, non-rigid objects, and objects in complex environments [53]. Figure 4 shows the calculation process of the convolution kernel. The offset is calculated for the input feature graph using the convolution layer and the offset has the same resolution as the output feature map. Figure 2: Residual network structure. (**a**) Identity residual. (**b**) Projection residual. For an input feature map \(x\) and an output feature map \(y\) in a conventional two-dimensional convolution, the mathematical model of the convolution can be expressed as Equation (6). \[y(P_{n})=\sum_{P_{n}R}W(P_{n})\times x(P_{0}+P_{n}) \tag{6}\]  
In the formula, \(R\) defines a \(3\times 3\) convolution kernel with an expansion rate of 1, which enumerates all positions in the output eigenmap. Where \(R=\{(1,1),(1,0),(1,1),\ldots,(1,-1)\), \((1,0),(1,1)\}\), \(W(\cdot)\) is the weight corresponding to the sampling point. In deformable convolution, the regular grid is extended by adding an offset (\(\{\Delta p_{n}|n=1,2,\ldots,n,N=|R|\}\)) and the filter deforms. The mathematical model of the deformed convolution can be expressed as Equation (7). \[y(p_{0})=\sum_{p_{n}\in R}w(p_{n})\times x(p_{0}+p_{n}+\Delta p_{n}) \tag{7}\]  
where \(n\) is the number of sampling locations and \(p_{-}0+p_{-}n\) is the offset of the sampling position. The process of deformable convolution is shown in Figure 3.  
Figure 4: Calculation process of deformable convolution. Figure 3: Comparison of deformable convolution sampling points of conventional \(3\times 3\) convolution kernel. (**a**) Conventional convolutional sampling position. (**b**) Random sampling of deformable convolutions. (**c**) Sampling position after deformable convolutional amplification. (**d**) Sampling position after deformable convolutional rotation. Since the offset is a decimal, it needs to be converted to an integer by a bilinear interpolation algorithm, as shown in Equation (8). \[x(p)=\sum_{q}G(q,p)\times x(q) \tag{8}\]  
where \(p\) represents any position after offset \((p=p_{0}+p_{n}+\Delta p_{n})\) and enumerates the spatial position of all feature maps \(x\). \(G(\cdot)\) is the kernel of bilinear interpolation. \(G\) is a two-dimensional vector, which can be expressed as the product of two one-dimensional vectors, as shown in Equation (9). \[G(q,p)=\text{g}(q_{x},p_{x})\times\text{g}\big{(}q_{y},p_{y}\big{)} \tag{9}\]  
where \(\text{g}(q_{x},p_{x})\) and \(\text{g}\big{(}q_{y},p_{y}\big{)}\) can be calculated by the following Equations (10) and (11). Figure 1:0  
Figure 1:1  
The offset can be learned by a backpropagation algorithm. In contrast to conventional standard convolutions, deformable convolutions contain the definition of standard convolution and can change and adjust their own shapes according to the demand of feature extraction, which provides more choices for the convolutional neural network in feature extraction, and has high adaptability to sample feature extraction [54]. Figure 5 shows the comparison between the feature point receptive fields of conventional convolution and deformable convolution. Take the defect (such as scratch) on the surface of an aluminum profile as an example. By comparison, it can be seen that the feature points of conventional convolution have a fixed size receptive field, while the deformable convolution can adaptively learn the sampling location of receptive field. Because the receptive field is more consistent with the shape and size of the object, it is more advantageous to feature extraction. (3) Improved ResNet network model  
Res-DBR (ResNet101 + Deformable conv + BN + ReLu) is a residual network module with deformable convolution. The module consists of a transformable convolutional layer, a BN layer, and an activation layer. 'Res' stands for ResNet101. 'DBR' stands for Deformable conv, BN, and ReLu.