## 4 Training Method Design
### Stages  
The proposed training method consists of the following stages:  
* self-supervision pre-training of the model with instance-prototype contrastive loss \(L_{ICPL}\);
* prototype and radius initialization for each class;
* supervised learning with loss function \(L_{S}\), which includes conventional cross-entropy and additional components for self-knowledge distillation and regularization;- selecting fault diagnostic data (selected randomly or with respect to the value of the loss function);
- inference on a new and diagnostic data;
- requesting manual labeling of hard examples;
- supervised learning with loss function \(\mathrm{L_{S}}\), taking into account manual labeling responses or semi-supervised fine-tuning (adaptation) with additional component \(\mathrm{L_{SSIN}}\) or \(\mathrm{L_{SSOUT}}\) depends on the result of inference;
- updating diagnostic data. Where a large amount of unlabeled data is available, the model preparation should start with self-supervised learning of feature extractor. For this purpose, it is proposed to use instance-prototype contrastive loss, characterized by computational efficiency and generalizing ability close to the biological prototype [53]. Instance-prototype contrastive loss is calculated by the formula  
\[\mathrm{L_{ICPL}}=-\log\frac{\exp(-\mathrm{dist}(\mathrm{z_{i}},\,\overline{ \mathrm{z_{i}}})/\tau)}{\exp(-\mathrm{dist}(\mathrm{z_{i}},\,\overline{ \mathrm{z_{i}}})/\tau)+\sum_{\mathrm{b=1}}^{\mathrm{b}}\exp(-\mathrm{dist}( \mathrm{z_{i}},\,\mathrm{z_{b}})/\tau)}, \tag{2}\]  
where \(\mathrm{z_{i}}\) is a feature representation at the output of the feature extractor \(\mathrm{z_{i}}=\mathrm{f}(\mathrm{x_{i}})\) for input example \(\mathrm{x_{i}}\) from a mini-batch; \(\overline{\mathrm{z_{i}}}\) represents an averaged feature representation for the augmented version of input example and considered as a positive pair for the input example \(\mathrm{x_{i}}\),  
\[\overline{\mathrm{z_{i}}}=\frac{1}{n_{\mathrm{a}}}\sum_{\mathrm{j=1}}^{n_{ \mathrm{a}}}\mathrm{f}\big{(}\mathrm{a_{j}}(\mathrm{x_{i}})\big{)}, \tag{3}\]  
where \(\mathrm{a_{j}}\) is an augmentation operator, such as random cropping, rescaling, random horizontal/vertical flip, random adjustment to hue, saturation, contrast and brightness, and random grayscale conversion;  
B is the number of pre-processed examples considered as negative pairs, the feature representations of which are stored in the non-indexed FIFO queue (with queue length \(\mathrm{B}=1024\)) and updated after the processing of each mini-batch;  
\(\tau\) is a temperature parameter that controls the dynamic range of the similarity function. The loss function \(\mathrm{L_{S}}\) used in supervised learning includes, in addition to the conventional cross-entropy \(\mathrm{L_{SCE}}\) computed with labels and predictions, a class-level component \(\mathrm{L_{CSD}}\) and a feature-level component \(\mathrm{L_{FSD}}\) of self-knowledge distillation. A regularization component \(\mathrm{L_{D}}\) can also be added to the loss function in order to achieve compression of feature representation, implementing an information bottleneck principle. The contrastive-center loss \(\mathrm{L_{CCL}}\) can be used as a regularization component to enhance robustness optimization by increasing intra-class compactness and inter-class separability.