# A Feature-Oriented Reconstruction Method for Surface-Defect Detection on Aluminum Profiles
## 3 Proposed Methods
### Image Essential Feature Learning Model
#### 3.2.3 Essential Features Extraction Network  
The structure of the essential features extraction model network, which is asymmetric, is shown in Figure 9. The encoder part firstly divides the input 224 \(\times\) 224 3D-area images into 16 \(\times\) 16 blocks after the feature-optimization layer and transforms it into a one-dimensional sequence. Next, it removes 75% of the random blocks by masking and introduces the Position and Class token encoding, and, finally, it outputs the encoder after the Transformer Block, which is made up of 24 Transformers stacked on top of each other,  
Figure 8: Feature-optimization module flowchart.  
is used for the feature learning, and is normalized by the Layer Norm. The decoder part takes the output of the encoder as input, introduces mask tokens and decoder-position-embedding encoding, which is repaired by a Transformer Block of 8 Transformer stacks, and finally outputs the same normalized by Layer Norm.  
Figure 9: Essential features extraction network.  
where \(X\in H\times W\times C\) and \(H\) and \(W\) represent the width and height of \(X\), respectively; \(C\) represents the number of channels; \(P^{2}\) represents the pixel size of the sequence; and \(P^{2}\cdot C\) represents the dimension of each sequence. (b) Positional Embedding: The positional encoding \(E_{pos}\) is introduced in order to prevent the loss of sequentiality of the positional information of the accession sequence, as shown in Equation (19). \[z_{1}=\left[x_{p}^{1}E;x_{p}^{2}E;\ldots;x_{p}^{n}E\right]+E_{pos}\] (19) (c) Class token: The Concat function is utilized to add a learnable category encoding \(x_{class}\) that is used to represent the global features of the image after encoding: \[z=\left[x_{class};x_{p}^{1}E;x_{p}^{2}E;\ldots;x_{p}^{n}E\right]+E_{pos}\] (20) (d) Transformer Block: The data \(Z\) is encoded through this process as in Equations (21) and (22). \[z_{n}^{\prime}=MHA(LN(Z_{n-1}))+Z_{n-1}\ (n=0,1,2\cdots,N)\] (21) \[z_{n}=MLP(LN(Z_{n}^{\prime}))+Z_{n}^{\prime}\ (n=0,1,2\cdots,N)\] (22) where \(MHA\) stands for Multiple Headed Attention, \(MLP\) stands for Multi-Layer Perceptual Machine, and \(n\) represents after n layers of transformer. (e) Layer Norm: The data output from (4) is normalized by this process as in Equation (23): \[y=\frac{x-E(x)}{\sqrt{Var(x)+\epsilon}}*\gamma+\beta\] (23) where \(E(x)\) denotes the mean, \(Var(x)\) denotes the variance, \(\epsilon\) the auxiliary variable, and the initial values of \(\gamma\) and \(\beta\) are 1 and 0, respectively. (f) Decoder: The output vector of the encoder is used as the input for the decoder, and after the data enters the decoder, it first passes through the Linear layer for dimensional conversion, and in order to ensure that it can distinguish between the different positions of mask tokens in the image, it will be added to the data as a whole with decoder-positional embedding. (g) Loss: The mean square error (\(MSE\)) and \(SSIM\)[38] are used as loss functions to calculate the loss of the original and restored images of the feature-optimization map as shown in (24), where the \(MSE\) expression is shown in (25): \[L_{PMRM}=(1-\alpha)MSE_{Y_{i},\hat{Y}_{i}}+\alpha(1-SSIM_{Y_{i},\hat{Y}_{i}})\] (24) \[MSE=\frac{1}{n}\sum_{i=1}^{n}\left(Y_{i}-\hat{Y}_{i}\right)^{2}\] (25) where \(Y_{i}\) denotes feature-optimized image original data, \(\hat{Y}_{i}\) denotes feature-optimized image reconstruction data, and \(\alpha\) denotes a weight factor that balances the relative importance between pixel and \(SSIM\), and in this paper, we set \(\alpha=0.5\).