# FGM: Noisy Label Learning via Fine-Grained Confidence Modeling
## 3 Proposed Method
### Generalization of Sample Clusters  
The process of generating sample clusters and performing cluster refinements is presented in Algorithm 1 which is located in page 6. Let \(\mathcal{S}=(\mathcal{X},\mathcal{AV})=\{(x_{i},ay_{i})\}_{i=1}^{N}\) denote the training set consists of \(N\) samples, where \(x_{i}\) is an input image with \(ay_{i}\) being the corresponding annotation label. To obtain the maximum repeating pseudo label \(my_{i}\) for sample \(x_{i}\), we need to record the history prediction confidence for every sample as \(C^{\epsilon_{k}}\). Given a model with parameter \(\theta\), the confidence score \(c_{i}^{\epsilon_{k}}(\tilde{y})\) represents the probability of sample \(i\) to be classified into class \(\tilde{y}\) after \(k\)th epoch of training:  
\[c_{i}^{\epsilon_{k}}(\tilde{y})=\mathcal{P}_{model}^{\theta}(x_{i};\theta_{ \epsilon_{k}}) \tag{1}\]  
where \(\mathcal{P}_{model}^{\theta}\) refers to the model's output softmax probability for class \(\tilde{y}\). We also maintain a history prediction list \(\mathcal{H}^{\epsilon_{k}}\) which contains the pseudo labels produced by the model for each sample from the beginning of the training process to the current epoch \(k\):  
\[\mathcal{H}^{\epsilon_{k}}=\{\{\arg\max\mathcal{P}_{model}(x_{i};\theta_{ \epsilon_{j}})\}_{i=1}^{N}\}_{j=0}^{k} \tag{2}\]  
where \(\arg\max\mathcal{P}_{model}(x_{i};\theta_{\epsilon_{j}})\) refers to the predicted label with maximum confidence, as known as the pseudo label, for sample \(i\) in epoch \(j\). We use the label with maximum number of occurrences in \(\mathcal{H}^{\epsilon_{k}}_{i}\) as \(my_{i}\) for sample \(i\). The model is trained on the whole training set in the early stage when the impact of noisy labels is minor. After the warmup phase ends, a joint confidence score \(jc_{i}^{\epsilon_{k-1}}\) is constructed for sample \(i\) in every epoch \(k\):  
\[jc_{i}^{\epsilon_{k-1}}=(\frac{\sum_{j=0}^{k-1}c_{i}^{\epsilon_{j}}(ay_{i})}{ k},\frac{\sum_{j=0}^{k-1}c_{i}^{\epsilon_{j}}(my_{i})}{k}) \tag{3}\]  
The two elements in \(jc_{i}^{\epsilon_{k-1}}\) represent the average of historical prediction confidence for annotation and maximum repeating pseudo label, respectively, reflecting the trends of their confidence scores during the training process. Gaussian Mixture Model (GMM) [29] is a widely used unsupervised modeling technique. A Gaussian Mixture Model is a weighted sum of \(M\) component Gaussian densities, where \(M\) is the number of clusters in the data. GMM assumes that the input data points are generated from a mixture of \(M\) Gaussian distributions. With parameter estimation using the Expectation-Maximization algorithm [30], GMM outputs the probabilities of samples belonging to each cluster. In this paper, we categorize samples into the cluster with maximum possibility. Owing to the different trends showed in Figure 1, five clusters of sample points can be generated by fitting a five-component Gaussian Mixture Model(GMM) [29] to \(jc^{\epsilon_{k-1}}\). The clustering result, presented in Figure 2b, is highly consistent with that in Figure 2a, which is produced by replacing \(my_{i}\) with \(gy_{i}\), indicating the effectiveness of using \(\mathcal{MY}\) as the alternative for \(\mathcal{GV}\). The cluster with the highest average confidence score of \(ay_{i}\) is CS, and the second-highest cluster is CH. The cluster with the maximum difference between the average scores of \(my_{i}\) and \(ay_{i}\) is NS. The other two clusters are NHR and NHI, whose labels are ambiguous and less valuable to training models. Moreover, the clusters can be further refined with priors as described in the next subsection. Consequently, we can obtain a new training set \(\mathcal{T}\) with the cleanest labels by screening out instances from NHR and NHI and rectifying NS samples' labels with their maximum repeating pseudo labels. It is worth noting that though NHR and NHI are not used in training, their existences improve the clustering performance.