## 3 Methodology
### Diffusion Pseudo-Label Matching  
Based on the obtained cluster labels \(\left\{y_{i}^{s}\right\}\) of \(\mathfrak{X}\) over \(K\) clusters, our goal is to match them with the ground-truth labels to guide target generation in the GDUI model. In unsupervised situations, we do not have ground truth to match against. To address the challenge of matching the ground-truth labels with the obtained cluster labels and to ensure a globally attentive alignment, we adopt the principles of the Stable Marriage Algorithm (SMA) [44] for the overall matching strategy. This approach emphasizes the importance of considering global information in the matching process. To address this issue, we propose the pseudo-label-matching algorithm, which leverages the zero-shot capability of CLIP to achieve bilateral matching between the clustering labels and the ground truth. Given the clustering probability matrix \(\mathbf{P}\) and \(K\) clusters with cluster labels \(\left\{y_{i}^{s}\right\}\), the top confident samples are selected as the clustering prototypes for each cluster. To illustrate the process, we take the \(m\)-th cluster as an example. We define the \(m\)-th cluster as:  
\[\mathfrak{X}_{m}=\left\{\left(\mathbf{x}_{i},y_{i}^{s}\right)\mid y_{i}^{s}=m, \forall i=1,2,\ldots,N\right\} \tag{10}\]  
The top samples are selected as:  
\[\mathfrak{X}_{m}^{top}=\left\{\left(\mathbf{x}_{i},y_{i}^{s}\right)\mid i\in \operatorname{argt}  
where the argtopk function \(\operatorname{argtopk}\left(\mathbf{P}_{j,m},N_{top}\right)\) returns the indices of the top \(N_{top}\) highest-scoring samples in the \(m\)-th cluster.  
Using CLIP, zero-shot classification is performed on the samples in \(\mathfrak{X}_{m}^{top}\) with respect to the provided ground-truth label set \(\mathcal{Y}^{g}=\left\{y_{i}^{g}\right\}_{i=1}^{K}\). The classes with the highest classification probability are then selected as the labels for the samples. We obtain \(\mathbf{p}_{i}^{c}\) as the CLIP classification probability for the \(m\)-th cluster by calculating the proportion of each class in the \(N_{top}\) samples. This ensures that the priority of each class in the clustering result is directly proportional to its probability such that higher probabilities correspond to higher priorities. Then, based on the previously obtained \(K\) clusters, the CLIP priority matrix for the \(K\) clusters, \(\mathbf{P}^{c}=\left[\mathbf{p}_{1}^{c},\mathbf{p}_{2}^{c},\ldots,\mathbf{p}_{K}^{c}\right]^{ \mathsf{T}}\in\mathbb{R}^{K\times K}\), is constructed.  
A cluster, for example, the \(m\)-th cluster \(\mathfrak{X}_{m}\), is selected from the unmatched clusters \(\mathfrak{U}=\left\{\mathfrak{X}_{i}\right\}_{i=1}^{K}\) that have not been matched with the ground-truth label set \(\mathcal{Y}^{g}\), and the highest priority class \(y_{i}^{g}\), represented as:  
\[y_{i}^{g}=\operatorname{argmax}\mathbf{P}_{m,j}^{c},j\in\mathcal{Y}_{m}^{u} \tag{12}\]  
where \(\mathcal{Y}_{m}^{u}\) represents the ground truth of the \(m\)-th cluster that has not been requested for label matching, and \(\mathbf{P}_{m,j}^{c}\) denotes the element in the \(m\)-th row and column j of matrix \(\mathbf{P}^{c}\), where j is the index of the element in the \(\mathcal{Y}_{m}^{u}\). Then, \(\operatorname{argmax}\mathbf{P}_{m,j}^{c}\) returns the index of the highest priority class among all unmatched ground truths for the \(m\)-th cluster. If the class \(y_{i}^{g}\) has not been assigned, then it is assigned to the current \(m\)-th cluster. Otherwise, its priority is compared with the already assigned clusters. The cluster with a lower priority is added back to the unmatched clustering set \(\mathfrak{U}\), while the one with a higher priority is matched with class \(y_{i}^{g}\). Until all clusters are matched, we can obtain each cluster and its corresponding label, denoted by \(\mathfrak{M}=\left\{\left(\mathfrak{X}_{i},y_{i}^{g}\right)\right\}_{i=1}^{K}\), where each label corresponds to the true class. The above process for diffusion pseudo-label matching is summarized in Algorithm 1.