# Learning Accurate Pseudo-Labels via Feature Similarity in the Presence of Label Noise
## 3 Materials and Methods
### Generate Feature Labels  
To address the aforementioned issues, we suggest utilizing a blend of the exponential moving average of the previous round of pseudo-labels, predicted labels, and feature labels to generate corrected pseudo-labels at the feature level. The objective of this strategy is to address the instability observed in the corrected labels during the initial training phases and ensure that the ground-truth class labels are appropriately weighted, even when the original labels contain noise. Additionally, due to the incorporation of feature clustering in our algorithm, there may be a slight increase in computational time. Initially, features, \(f\), are extracted from all samples using the feature extractor, \(F\). Given that the extracted features might vary in magnitude, we normalize the features based on their respective classes to obtain normalized features, \(\widehat{f}_{i}\). \[\widehat{f}_{i}=\frac{f_{i}}{\|f_{i}\|} \tag{3}\]  
This step is performed to facilitate the analysis of the raw feature data. We calculate the feature center, \(O_{c}\), for each class based on \(f_{i}\), as follows:  
\[O_{c}=\frac{\sum_{i=1}^{N_{c}}\widehat{f}_{i}}{\|\sum_{i=1}^{N_{c}}\widehat{f} _{i}\|} \tag{4}\]  
Here, \(c\) takes values from the set \(\{1,2,3,\dots,C\}\), denoting the specific class. C signifies the overall count of classes, while \(N_{c}\) denotes the total samples, where \(s_{i}=c\); this indicates the count of samples with a pseudo-label class, \(c\). \(\widehat{f}_{i}\) denotes the normalized feature of the \(i\)-th sample within the respective class. In addition, to acquire feature labels, we compute the cosine similarity between each sample and the centroids of each feature class, denoted as \(COS_{ij}\). \[COS_{ij}=\widehat{f}_{i}\cdot O_{j} \tag{5}\]  
where \(j\in\{1,2,3,\dots,C\}\) represents the corresponding class number. \(COS_{ij}\) represents the cosine similarity between the \(i\)-th sample and the feature centroid, \(O_{j}\), of the \(j\)-th class. Following this, we establish the feature label, \(u_{i}\), for each sample by evaluating its cosine similarity with the feature centers of each class. A greater cosine similarity suggests that the sample pertains to the class housing the respective feature center. Thus, we define the criterion as follows:  
\[u_{i}=argmax(COS_{i1},COS_{i2},COS_{i3}\dots COS_{ic}) \tag{6}\]  
In this case, \(argmax\) identifies the index of the highest value in a given set of operations. For example, if the value of \(COS_{i3}\) is the highest, the returned value would be 3. In other words, \(u_{i}\) would be equal to 3. For each sample, we compute its cosine similarity to the centroid of each class. The higher the similarity to a centroid, the higher the probability of belonging to that class. By extracting features and calculating similarities, samples are assigned pseudo-labels based on the classification results, making the relationship between samples and their features more pronounced, extending beyond simple sample label predictions. To rectify noisy labels for a larger sample set and render them viable, we employ pseudo-label updating. Through the utilization of an exponential moving average scheme, we progressively rectify noisy labels by model predictions and feature labels. This strategy aids in mitigating the volatility of model predictions, smooths the training process, and facilitates the complete overhaul of labels for training samples when necessary. In generating pseudo-labels, we extract features, \(f\), from a small data batch and feed them into a classifier to derive predictions, \(p\). Subsequently, the pseudo-labels, \(s_{i}\), are adjusted based on the prediction outcomes and feature labels:  
\[s_{i}=\alpha s_{i-1}+\beta p_{i}+\gamma u_{i} \tag{7}\]Here, \(\alpha\), \(\beta\), and \(\gamma\) are hyperparameters, with the constraint that \(\alpha+\beta+\gamma=1\). In the next section of the experiment, we will provide different hyperparameter settings for different datasets.