# An Improved Heteroscedastic Modeling Method for Chest X-ray Image Classification with Noisy Labels
## 3 Methodology
### The Proposed GMM-HM
#### 3.2.2 Noisy Detector  
Generally, the small-loss samples are considered as the clean ones, and the large-loss samples as the noisy ones [32; 33]. We could straightforwardly use the loss values to distinguish the clean and the noisy samples by setting a threshold. However, the threshold is difficult to choose for different datasets. Another way is to model the loss distribution and then distinguish the clean or noisy samples by the distribution. [PERSON] et al. [33] propose that due to the flexibility of the Gaussian Mixture Model (GMM) model distribution, clean and noise samples can be better distinguished. To this end, we build a GMM based on the losses to determine whether one sample is clean or noisy.  
Given training data \(\mathcal{D}=\{\mathcal{X},\mathcal{Y}\}=\{(x_{i},y_{i})\}_{i=1}^{N}\), where \(N\) is the total number of training images, \(x_{i}\) is the i-th training image and \(y_{i}=[y_{i}^{\prime},y_{i}^{\prime},...,y_{i}^{\prime}]\) is its label vector. \(\mathcal{C}\) is the number of the pathologies. If \(x_{i}\) belongs to class \(c\), then \(y_{i}^{\prime}=1\); otherwise, \(y_{i}^{c}=0\). Let \(\mathcal{L}(\mathcal{X})\) be the loss values at the epoch of producing the best performance. We build a \(K\)-component GMM to loss values \(\mathcal{L}(\mathcal{X})\), and the predicted probability of each sample is formalized as  
\[p(g_{k}|\mathcal{L}(X))=\sum_{k=1}^{K}\phi_{k}\frac{1}{\sqrt{2\delta_{k}^{2}} \pi}e^{-\frac{(\mathcal{L}(X)-x_{k})^{2}}{2\delta_{k}^{2}}}, \tag{4}\]  
where \(v_{k}\) and \(\delta_{k}\) are the mean and standard variance of the \(k\)-th Gaussian component, respectively. \(\phi_{k}\) is the weight of the \(k\)-th Gaussian component. \(K\) is 2 in our experiments. The parameters \(v_{k}\), \(\delta_{k}\), and \(\phi_{k}\) of the GMM can be estimated using the Expectation Maximization algorithm. At each epoch, GMM divides the training data into a clean set and a noisy set. Specifically, we distinguish one sample as belonging to clean set (\(\mathcal{S}_{clean}\)) or noisy set (\(\mathcal{S}_{noisy}\)) by setting a threshold \(T\) on the \(p(g|L(x_{i}))\),  
\[x_{i}\in\begin{cases}\mathcal{S}_{clean}&p(g|\mathcal{L}(x_{i}))<T\\ \mathcal{S}_{noisy}&p(g|\mathcal{L}(x_{i}))\geq T,\end{cases} \tag{5}\]  
where \(g\) is the Gaussian component with a smaller mean (small loss). Generally, we set the \(T\) as 0.5 in the experiment. At the first epoch, due to the fact that there are effective loss values, we consider all the training samples as clean ones, feeding into the clean branch of the noise-aware classification network. In subsequent training, we dynamically fit a two-component GMM on its per-sample loss distribution in the previous epoch to divide the training samples into a noisy and a clean set. Whether a sample belongs to the clean or noisy set is determined by the posterior probability \(p(g|\mathcal{L}(x_{i}))\) whose loss belongs to the Gaussian component with smaller mean (smaller loss). That is, if \(p(g|\mathcal{L}(x_{i}))\) is larger than a threshold T, the corresponding sample is judged to the clean set. Otherwise, it would be classified as noisy. Therefore, all the samples in the noisy set would input into the noisy branch of the noise-aware classification network from the second epoch in the training.  
Figure 2: The framework of the proposed GMM-HM.