## 2 Methods
### Momentum Contrastive Learning  
For large-scale unlabeled WBMs, the momentum contrastive learning method is introduced in this study for self-supervised learning. Momentum contrastive learning is designed to improve the training efficiency of contrastive learning algorithms by introducing a momentum-based update to the model weights, and it is effective in various applications, such as image classification, object detection, and natural language processing. It is advantageous when labeled data are limited or unavailable, as it generates high-quality feature representations without manual annotation.  
The proposed momentum contrastive learning framework is presented in Figure 4. Contrastive learning [32] can be regarded as training an encoder for a dictionary look-up task. It generates two augmented views for each WBM in a batch randomly sampled from a WBM dataset. These two views are then projected into an embedding space, where a self-supervised loss is applied to train the encoder and generate effective representations.  
Figure 3: The proposed two-stage flowchart.  
Momentum contrastive learning consists of three parts. The first part is the data augmentation module, which generates two correlated views of each WBM. To facilitate WBM defect classification, we apply simple augmentations such as rotation and flipping. These augmentations are designed to preserve the underlying failure patterns and do not alter the latent categories of the data.  
The second part is the embedding generation part. The input of this part is the query (one augmented WBM) and the key (another augmented WBM) of each WBM. Two encoders transform the WBMs from two augmented views to embedding. As ResNet has been shown to achieve state-of-the-art results on various computer vision tasks, we applied ResNet-18 [39] as the backbone of the two encoders.  
Moreover, the queue is a significant development of momentum contrastive learning, shown in the second part. Contrastive learning involves constructing a discrete dictionary from high-dimensional inputs representing a sampled subset of all data. In momentum contrastive learning, the dictionary is maintained as a queue of data embedding. Using a queue, it decouples the dictionary size \(K\) from the batch size, which can be much larger than the batch size. Additionally, the queue is renewed after each epoch. At the end of each training episode, the current batch is enqueued to the dictionary, and the oldest batch in the queue is removed to maintain a fixed queue size.  
The third part is contrastive loss calculation and parameter update. The queries and their corresponding keys are encoded for the current batch, forming positive sample pairs. The negative samples are all the samples in the queue. A contrastive loss is a function whose value is low when the query is similar to its corresponding positive key and dissimilar to all other keys in the queue. The contrastive loss function is InfoNCE [32]:  
\[L_{q}=-\log\frac{\exp(q\cdot k_{+}/\tau)}{\sum_{i=0}^{K}\exp(q\cdot k_{i}/ \tau)}, \tag{1}\]  
where \(q\) is the query and \(k_{+}\) is the positive key of \(q\). \(k_{i}\) represents the keys in the queue.  
The second development of momentum contrastive learning is the momentum update. In the backpropagation period, the gradient only propagates to the queries, and only the parameters of the query encoder are updated by backpropagation. Then, the parameters of the key encoder are updated by:  
\[\theta_{k}\gets m\theta_{k}+(1-m)\theta_{q} \tag{2}\]  
Here, \(m\) is a momentum coefficient. \(\theta_{k}\) and \(\theta_{q}\) represent the parameters of the momentum encoder and query encoder, respectively. We default to \(m=0.999\) because a relatively  
Figure 4: The proposed momentum contrastive learning framework.  
large momentum works much better than a smaller value. It is suggested that a slowly evolving key encoder is core to using a queue [32].