## 3 Methodology
### Diffusion Label Matching Refinement  
In the SPICE framework, an imperfect feature model can cause similar features to be assigned to truly different clusters, while imperfect cluster heads can result in dissimilar samples being assigned the same cluster label. These issues may eventually lead to the presence of samples from different classes in the same cluster and mismatches between samples and their true labels. Alternatively, these errors can also be caused by the pseudo-label-matching algorithm, which can result in mismatches between the cluster labels and the true labels of the clusters they represent. To overcome these issues, we propose a diffusion model label-matching refinement algorithm to adjust the matching of labels within clusters. Here, we also use the \(m\)-th cluster as an example, similar to our previous selection of \(\mathfrak{X}_{m}^{top}\) and the least confident samples \(\mathfrak{X}_{m}^{btm}\) for \(m\)-th cluster being selected as:  
\[\mathfrak{X}_{m}^{btm}=\{(\mathbf{x}_{i},y_{i}^{s})\ |\ i\in\text{arglowk}(\mathbf{P}_{ j,m},N_{btm}),\mathbf{x}_{j}\in\mathfrak{X}_{m}\} \tag{13}\]  
where the arglowk function \(\text{arglowk}(\mathbf{P}_{j,m},N_{btm})\) returns the indices of the least \(N_{btm}\) confident samples, selected from the indices belonging to the \(m\)-th samples in the \(m\)-th column of matrix \(\mathbf{P}_{:,m}\). Similar to \(\mathfrak{X}_{m}^{top}\), zero-shot classification based on true labels for \(\mathfrak{X}_{m}^{btm}\) is also performed using CLIP. Furthermore, the semantic matching ratios \(\delta_{m}^{top}\) and \(\delta_{m}^{btm}\) for the top and bottom of the \(m\)-th cluster can be represented as:  
\[\delta_{m}^{top}=\frac{1}{N_{top}}\sum_{y\in\mathfrak{X}_{m}^{top}}1\left(y=y _{i}^{s}\right) \tag{14}\]  
\[\delta_{m}^{btm}=\frac{1}{N_{btm}}\sum_{y\in\mathfrak{X}_{m}^{btm}}1\left(y=y _{i}^{s}\right) \tag{15}\]  
where \(\delta_{m}^{top}\) and \(\delta_{m}^{btm}\) reflect the matching status of the top and bottom of the \(m\)-th cluster. To comprehensively reflect the matching status of the \(m\)-th cluster, the overall semantic matching ratio \(\delta_{m}\) is defined as:  
\[\delta_{m}=\delta_{m}^{top}*w_{top}+\delta_{m}^{btm}*w_{btm} \tag{16}\]  
where \(w_{top}\) and \(w_{btm}\) represent the weights of \(\delta_{m}^{top}\) and \(\delta_{m}^{btm}\), respectively, in the overall matching ratio \(\delta_{m}\). If the overall semantic matching ratio \(\delta_{m}>\lambda_{\delta}\), where \(\lambda_{\delta}\) is the overall matching threshold, then a high matching degree for the \(m\)-th cluster \(\mathfrak{X}_{m}\) implies that the cluster label \(y_{m}^{s}\) for that is trustworthy. In other cases, further examination is required to determine the matching status of the top and bottom of the \(m\)-th cluster. In cases where the matching status of the top \(\delta_{m}^{top}\) in the \(m\)-th cluster is greater than the top matching threshold \(\lambda_{top}\) but the matching status of the bottom \(\delta_{m}^{btm}\) is less than the bottom matching threshold \(\lambda_{btm}\), it is necessary to evaluate the semantic consistency ratio \(r_{m}^{btm}\) of the least confident samples \(\mathfrak{X}_{m}^{btm}\), which can be defined as:  
\[r_{m}^{btm}=\frac{1}{N_{btm}}\sum_{\mathfrak{X}_{i}\in\mathfrak{X}_{m}^{btm}} \mathbf{P}_{i,m} \tag{17}\]  
where \(\mathbf{P}_{i,m}\) denotes the clustering probability of the element located in the \(i\)-th row and \(m\)-th column of matrix \(\mathbf{P}\), specifically referring to the probability that \(x_{i}\in\mathfrak{X}_{m}^{btm}\) belongs to the \(m\)-th cluster \(\mathfrak{X}_{m}\). When \(r_{m}^{btm}\) exceeds the confidence threshold \(\sigma_{btm}\), it implies that even if the matching degree at the bottom level is lower than the threshold, the overall consistency of the \(m\)-th cluster \(\mathfrak{X}_{m}\) is sufficiently reliable, thus suggesting that the clusterlabel \(y_{m}^{g}\) as a whole can be trusted.