## 3 Materials and Methods
### SimSiam  
We employed SimSiam [29] for pretraining a feature extractor specifically tuned to discriminate between relevant features from manufacturing images. SimSiam was chosen for its optimal trade-off between simplicity and effectiveness in image classification tasks. Its advantage lies in not requiring large batch sizes or a momentum encoder for training, making it feasible to train on multiple GPUs without demanding significant resources. SimSiam is a contrastive learning approach that employs a Siamese-network-based architecture. This method trains the model to maximize the agreement between different augmented views of the same data using contrastive loss in the latent space. It generates two random augmentations, \(x_{1}\) and \(x_{2}\), from an image \(x\) which are then processed by an encoder network \(f\). This encoder \(f\), comprising a backbone such as ResNet50 [45] and a projection multilayer perceptron (MLP) head, shares its weights across views for consistent feature extraction. Additionally, a separate MLP, denoted as \(h\), serves as a prediction head,  
Figure 1: Methodology overview: The process comprises two stages, pretraining and fine-tuning. Initially, we utilize self-supervised learning techniques SimSiam and SimMIM to train the generic backbone on large volumes of unlabeled X-ray images. This initial phase helps the model to capture critical visual features relevant to manufacturing defects, eliminating the need for data annotations. Subsequently, the pretrained backbone is integrated into an object detection framework, where it undergoes fine-tuning with an annotated target dataset. The final inference stage applies the fine-tuned model to accurately detect defects in manufacturing images, using the robust features learned in earlier phases for improved precision and reliability. transforming one view's output to align with the other view. Following the notation of the original paper, the transformed and original vectors are represented as \(p_{1}\triangleq h(f(x_{1}))\) and \(z_{2}\triangleq f(x_{2})\), respectively. In a symmetrized manner, the model also processes the augmentations inversely, producing \(z_{1}\triangleq f(x_{1})\) and \(p_{2}\triangleq h(f(x_{2}))\). This symmetrization in the loss function significantly enhances the robustness and learning effectiveness of the model. The core objective of SimSiam is to minimize the negative cosine similarity between pairs (\(p_{1}\) and \(z_{2}\) and \(p_{2}\) and \(z_{1}\)), thereby aligning these representations despite their augmentation-induced variations. \[ \begin{split}\mathcal{D}(p_{1},z_{2})=&-\frac{p_{1} }{\|p_{1}\|_{2}}\cdot\frac{z_{2}}{\|z_{2}\|_{2}}\\ \mathcal{D}(p_{2},z_{1})=&-\frac{p_{2}}{\|p_{2}\|_{ 2}}\cdot\frac{z_{1}}{\|z_{1}\|_{2}}\end{split}  \tag{1}\]  
where \(\|p_{1}\|_{2}\) and \(\|p_{2}\|_{2}\) are the L2 norms of \(p_{1}\) and \(p_{2}\), respectively. A pivotal element of the SimSiam approach is the stop-gradient operation, which is key to preventing a collapsing solution where all outputs converge to the same vector. This operation is applied to the output of the encoder \(f\), treating it as a constant during a portion of the training. This technique helps maintain diversity in the learned features, ensuring that the model does not trivially minimize the loss by converging to a constant output, as illustrated in Figure 2. The final form of the loss function is defined as follows:  
\[\mathcal{L}=\frac{1}{2}\mathcal{D}(p_{1},stopgrad(z_{2}))+\frac{1}{2}\mathcal{ D}(p_{2},stopgrad(z_{1})) \tag{2}\]  
The goal is for the model to learn representations that are invariant to the augmentations, ensuring the model yields consistent outputs for different views of the same image. This necessitates the careful selection of augmentations that are sufficiently invariant for trivial variations to be ignored, but not excessively, so important information for downstream tasks is not discarded [46]. Achieving this balance enables the model to effectively identify and highlight relevant features.