SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Advantages over Distance Calibration Firstly, our approach can be applied to a wide range of kernels and yields an improved kernel matrix. In contrast, distance calibration methods only benefit distance-based kernels and lack quality guarantees. Moreover, our algorithm, which corrects the Gaussian kernel, can generate a high-quality distance matrix via  $d_{ij} = \sqrt{-\sigma^2\log(k_{ij})}$ . By incorporating the second-order term of  $d_{ij}^{0}$  in  $k_{ij}^{0} = \exp(-(d_{ij}^{0})^{2}/\sigma^{2})$ , our algorithm becomes more sensitive to changes in distance values, while the Euclidean embedding algorithm relies on  $K^{0} = \exp(-\gamma D^{0})$  using the first-order term of  $d_{ij}^{0}$ . Empirical evidence suggests that the corrected distance obtained from our corrected Gaussian kernel is more accurate than the calibrated distance derived from the Euclidean embedding method, benefiting to the distance-based spectral clustering.
Benefits to Spectral Clustering Our approach is tailored to improve the kernel quality, making it highly advantageous for spectral clustering. This improvement benefits both standard spectral clustering using the Gaussian kernel and affinity learning based on the self-expressive framework. In the case of  $X$ -based self-expressive affinity, we can apply the corrected linear kernel  $K \coloneqq X^{\top}X$  to the optimization, i.e.,  $\min_{C}\frac{1}{2}\|X - XC\|_F^2 + \lambda \mathcal{R}(C) = \min_{C}\frac{1}{2}\mathrm{Tr}(K - 2KC + C^{\top}KC) + \lambda \mathcal{R}(C)$ . Similarly, for  $K$ -based self-expressive affinity, which involves using the Gaussian kernel or other kernels, our approach can be applied to  $K \coloneqq \phi(X)^{\top}\phi(X)$  on  $\min_{C}\frac{1}{2}\|\phi(X) - \phi(X)C\|_F^2 + \lambda \mathcal{R}(C)$ .
4 Methodology-II. Affinity Learning
Taking advantage of the corrected kernel matrix, we have further designed a series of kernel-based affinity learning algorithms to acquire high-quality affinity matrices with an adaptive extension.
4.1 Kernel Self-expressive Learning with  $\ell_p$ -norm
We utilize the kernel self-expressive framework, i.e.,  $\min_C\frac{1}{2}\| \phi (X) - \phi (X)C\| _F^2 +\lambda \mathcal{R}(C)$ , and propose new affinity learning methods by enhancing the sparsity of the affinity matrix  $C$  using the  $\ell_p$  norm [60]. Specifically, our approach diverges from previous work [19, 20, 3] by incorporating two forms of  $\ell_p$ -norm, i.e., proximal  $p$ -norm and Schatten  $p$ -norm  $(0 < p < 1)$ , as regularization terms.
Definition 4 (Kernel Self-expressive Learning with Proximal  $p$ -norm).
$$
\underset {C \in \mathbb {R} ^ {n \times n}} {\text {m i n i m i z e}} \frac {1}{2} \| \phi (X) - \phi (X) C \| _ {F} ^ {2} + \frac {\lambda}{2} \| C \| _ {p} ^ {p}, \text {s u b j e c t t o} 0 \leq c _ {i j} \leq 1, \forall 1 \leq i, j \leq n, \tag {7}
$$
where the proximal  $p$ -norm is expressed as  $\|C\|_p^p \coloneqq \sum_{i,j=1}^{n} |c_{ij}|^p$ .
We construct an augmented Lagrangian function  $\mathcal{L}_p(C,Z,U)$  in Eq. (8), and solve it by using the Alternating Direction Method of Multipliers (ADMM) approach [61] summarized in Algorithm 2.
$$
\mathcal {L} _ {p} = \| \phi (X) - \phi (X) C \| _ {F} ^ {2} + \lambda \| Z \| _ {p} ^ {p} + \gamma \sum_ {i, j} \max  \left(z _ {i j} - 1, 0\right) ^ {2} + \operatorname {T r} \left(U ^ {\top} (Z - C)\right) + \frac {1}{2 \rho} \| Z - C \| _ {F} ^ {2}. \tag {8}
$$
However, the KSL-Pp algorithm involves numerous hyper-parameters and entails a non-convex optimization process during the  $Z$ -update step (Line 5 in Algorithm 2), making it difficult to effectively utilize. Computation speedup needs further investigations. Thus, we focus on the Schatten  $p$ -norm.
Algorithm 2 Kernel Self-expressive Learning with Proximal  $p$ -norm (KSL-Pp)
Input:  $K$  : a kernel;  $\lambda ,p,\rho ,\gamma ,\alpha$  : hyperparameters; maxiter: maximum iterations; tol: tolerance.   
Output:  $\hat{C}\in \mathbb{R}^{n\times n}$  : the optimal affinity matrix of Eq. (7). 1: Initialize  $C_0,Z_0,U_0$  Refer to Appendix B.1 for formula derivations.   
2: for  $t = 0,1,\ldots$  ,maxiter do   
3:  $C$  -update:  $C_{t + 1}\gets (2K + \frac{1}{\rho} I)^{-1}(2K + U_t + \frac{1}{\rho} Z_t)$    
4:  $Z$  -update:  $Z_{t + 1}\leftarrow \frac{\partial\mathcal{L}_p}{\partial z_{ij}} = \lambda pz_{ij}^{p - 1} + \gamma \mathbb{I}_{(z_{ij} > 1)} + \frac{1}{\rho} (z_{ij} - c_{ij}) + u_{ij} = 0;$    
5:  $U$  -update:  $U_{t + 1}\gets U_t + \rho (Z_{t + 1} - C_{t + 1})$    
6: if (primal)  $\| C_{t + 1} - z_{t + 1}\| _F\leq tol$  and (dual)  $\| Z_{t + 1} - Z_t\| _F\leq tol$  then; break; end if.   
7: end for   
8: Set  $\hat{C} = (|C_t| + |C_t^\top |) / 2$
Definition 5 (Kernel Self-expressive Learning with Schatten  $p$ -norm).
$$
\underset {C \in \mathbb {R} ^ {n \times n}} {\text {m i n i m i z e}} \frac {1}{2} \| \phi (X) - \phi (X) C \| _ {F} ^ {2} + \frac {\lambda}{2} \| C \| _ {S _ {p}}, \text {s u b j e c t t o} 0 \leq c _ {i j} \leq 1, \forall 1 \leq i, j \leq n, \tag {9}
$$
where Schatten  $p$ -norm  $\| C\|_{S_p} \coloneqq (\sum_{i = 1}^n\sigma_i^p (C))^{1 / p}$  and  $\sigma_{i}(C)$  denotes the  $i$ -th singular value of  $C$ .
Drawing from prior research [62], for  $\frac{1}{2} < p < 1$  and  $k \geq \mathrm{rank}(C)$ , the following always holds true:
$$
\| C\|_{S_{p}} = \min_{U\in \mathbb{R}^{n\times k},V\in \mathbb{R}^{n\times k},C = UV^{\top}}\frac{\|U\|_{F}^{2} + \|V\|_{F}^{2}}{2}.
$$
Thus, for  $\frac{1}{2} < p < 1$ , we define  $\mathcal{L}_{S_p}(U,V) = \| \phi (X) - \phi (X)UV^\top \| _F^2 +\frac{\lambda}{2}\| U\| _F^2 +\frac{\lambda}{2}\| V\| _F^2$  as a relaxation of the optimization problem in Eq. (9) without loss of generality. By employing the ADMM approach [61] and the gradient descent method [63] in the  $U$ -update step (i.e., Line 4 in Algorithm 3), we design the KSL-Sp algorithm as follows with details in the Appendix B.2.
Algorithm 3 Kernel Self-expressive Learning with Schatten  $p$ -norm (KSL-Sp)
Input:  $K$  : a kernel matrix;  $\lambda$  : a hyperparameter; maxiter: maximum iterations; tol: tolerance.   
Output:  $\hat{C}\in \mathbb{R}^{n\times n}$  : the optimal affinity matrix of Eq. (9). 1: Initialize  $U_{0},V_{0}$  Refer to Appendix B.2 for formula derivations.   
2: for  $t = 0,1,\ldots ,$  maxiter do   
3:  $U$  -update:  $U_{t + 1}\gets \partial_U\mathcal{L}_{S_p} = 2K(UV_t^\top -I)V_t + \lambda U = 0;$    
4:  $V$  -update:  $V_{t + 1}\leftarrow 2KU_{t + 1}(2U_{t + 1}^\top KU_{t + 1} + \lambda I)^{-1}$  by  $\partial_V\mathcal{L}_{S_p} = 0$    
5: if  $\| U_{t + 1} - U_t\| _F <   tol$  then; break; end if.   
6: end for   
7: Set  $C_t = U_tV_t^\top$  and  $\hat{C} = (|C_t| + |C_t^\top |)\bigg{/}2.$
4.2 Adaptive Kernel Self-expressive Learning
To achieve adaptive affinity learning on incomplete data, we formulate a new joint optimization problem by incorporating KLSR [3] and kernel correction to learn kernel and affinity matrices iteratively with the PSD constraint of  $K$ :
$$
\underset {K \succeq 0, C \in \mathbb {R} ^ {n \times n}} {\text {m i n i m i z e}} \| K - K ^ {0} \| _ {F} ^ {2} + \operatorname {T r} (K - 2 K C + C ^ {\top} K C) + \lambda \| C \| _ {F} ^ {2}. \tag {10}
$$
Firstly, we introduce an auxiliary variable  $A \in \mathbb{R}^{n \times n}$  into
$$
\underset {K \succeq 0, A, C \in \mathbb {R} ^ {n \times n}} {\text {m i n i m i z e}} \| K - K ^ {0} \| _ {F} ^ {2} + \operatorname {T r} (A - 2 A C + C ^ {\top} A C) + \lambda \| C \| _ {F} ^ {2}, \quad \text {s u b j e c t t o} K = A, \tag {11}
$$
and then we derive the augmented Lagrange function  $\mathcal{L}(K,A,C,U)$  in preparation for ADMM:
$$
\mathcal {L} = \| K - K ^ {0} \| _ {F} ^ {2} + \operatorname {T r} (A - 2 A C + C ^ {\top} A C) + \lambda \| C \| _ {F} ^ {2} + \operatorname {T r} (U ^ {\top} (K - A)) + \frac {\rho}{2} \| K - A \| _ {F} ^ {2}, \tag {12}
$$
where  $U$  is a Lagrange multiplier and  $\rho$  is the updating step size, finally arriving at the Algorithm 4.
Our adaptive learning framework could be combined with other kernel self-expressive learning algorithms; however, due to space constraints, further research is required to explore these possibilities:
$$
\underset {K \succeq 0, C \in \mathbb {R} ^ {n \times n}} {\text {m i n i m i z e}} \| K - K ^ {0} \| _ {F} ^ {2} + \lambda_ {1} \| \phi (X) - \phi (X) C \| _ {F} ^ {2} + \lambda_ {2} \mathcal {R} (C), \text {w h e r e} K := \phi (X) ^ {\top} \phi (X). \tag {13}
$$
Algorithm 4 Adaptive Kernel Least-Squares Representation (AKLSR)
Input:  $K^0$  : an initial kernel;  $\lambda ,\rho$  : hyperparameters; maxiter: maximum iterations; tol: tolerance.   
Output:  $\hat{C}\in \mathbb{R}^{n\times n}$  : the optimal affinity matrix of Eq. (10). 1: Initialize  $K_{0},A_{0},C_{0},U_{0}$  Refer to Appendix B.3 for formula derivations.   
2: for  $t = 0,1,\ldots$  , maxiter do   
3:  $K$  -update:  $K\gets \frac{1}{\rho + 2} (2K^0 -Ut + \rho A_t)$  by  $\frac{\partial\mathcal{L}}{\partial K} = 0$    
4:  $K$  -PSD:  $K_{t + 1}\gets \mathcal{P}_1(K)$  by projecting  $K$  onto the PSD set via Eq. (6);   
5:  $A$  -update:  $A_{t + 1}\gets \frac{1}{\rho} (\rho K_{t + 1} + U_t + 2C_t^\top -C_ct_\top^- - I)$  by  $\frac{\partial\mathcal{L}}{\partial A} = 0$    
6:  $C$  -update:  $C_{t + 1}\gets 2(2\lambda I + A_{t + 1} + A_{t + 1}^\top)^{-1}A_{t + 1}^\top$  by  $\frac{\partial\mathcal{L}}{\partial C} = 0$    
7:  $U$  -update:  $U_{t + 1}\gets U_t + \rho (K_{t + 1} - A_{t + 1})$    
8: if  $\| C_{t + 1} - C_t\| _F <   tol$  then; break; end if.   
9: end for   
10: Set  $\hat{C} = (|C_t| + |C_t^\top |) / 2.$
We evaluate the performance on four benchmark datasets, including two face image datasets Yale64 [64] and Umist [65], a handwritten digit image dataset USPS [66], and a speech dataset Isolet [67]. We use a subset of USPS with 1,000 randomly selected samples. All experiments  $^2$  are conducted five times in MATLAB on a ThinkStation with a 2.1 GHz Intel i7-12700 Core and 32GB RAM.
Various methods dealing with incomplete data are considered for comparison: 1) statistical imputation: ZERO, MEAN,  $k$ -nearest neighbors ( $kNN$ ) [26], Expectation Maximization (EM) [68]; 2) matrix completion: Singular Value Thresholding (SVT) [27], Grassmanian Rank-one Update Subspace Estimation (GR) [28], Kernelized Factorization Matrix Completion (KFMC) [29]; 3) distance calibration: Double-Centering (DC) [38], Triangle Fixing (TRF) [35], Euclidean Embedding (EE) [39]. Our KC method uses the spectral decomposition (SD). Details are provided in Appendix C.
5.1 Validation of Kernel Correction on Gaussian Kernel and Euclidean Distance
When handling incomplete data that is missing completely at random, the quality of the estimated Gaussian kernel matrix and Euclidean distance matrix is measured using the relative-mean-square error  $(\mathbf{RMSE} = \frac{\|\hat{A} - A^{*}\|_{F}^{2}}{\|A^{0} - A^{*}\|_{F}^{2}})$  and the relative error  $(\mathbf{RE} = \frac{\|\hat{A} - A^{*}\|_{F}}{\|A^{*}\|_{F}})$ , where  $A^0$  is the naive kernel/distance matrix obtained by Eq. (1),  $\hat{A}$  is the estimated one using different methods, and  $A^{*}$  as the ground-truth. All Gaussian kernels are calculated by  $k_{ij} = \exp (-d_{ij}^2 /\sigma^2)$ , where  $\sigma =$  median  $\{d_{ij}\}$ . Our KC algorithm yields Gaussian kernel and Euclidean distance matrices with the lowest RMSEs and REs, validating the theoretical guarantee of kernel matrices in Theorem 1, as demonstrated in Table 1. Furthermore, the accuracy of the top-10 nearest neighbors is evaluated using Recall values [69]. The KC algorithm achieves the highest Recalls with improved local relationships, which in turn benefits spectral clustering, particularly for kNN graphs. More numerical results are in Appendix D.
Table 1: Quality comparisons of Gaussian kernel and Euclidean distance under a missing ratio  ${80}\%$  .
Dataset	Metric	Naive	ZERO	MEAN	kNN	EM	SVT	GR	KFMC	DC	TRF	EE	KC
Umist	RMSE-K↓	1.000	2.849	1.535	2.451	1.527	2.849	1.797	3.818	1.748	0.923	0.438	0.382
RMSE-D↓	1.000	20.80	27.64	21.20	27.62	20.80	8.397	14.05	110.4	0.916	0.440	0.431
RE-K↓	0.189	0.319	0.234	0.295	0.233	0.319	0.253	0.369	0.250	0.181	0.125	0.117
RE-D↓	0.107	0.487	0.561	0.492	0.561	0.487	0.309	0.400	1.122	0.102	0.071	0.070
Recall↑	0.726	0.092	0.171	0.119	0.172	0.092	0.596	0.248	0.226	0.740	0.771	0.785
$^{2}$ Codes are available at https://github.com/SciYu/Spectral-Clustering-on-Incomplete-Data.
5.2 Comparative Studies on Standard Spectral Clustering
We adopt the Gaussian kernel for standard spectral clustering and obtain its  $k\mathrm{NN}$  graph  $(k = 10)$ , as the input for clustering via the normalized cut algorithm [1]. We evaluate the clustering performance using Accuracy (ACC), Normalized Mutual Information (NMI), Purity (PUR), and Adjusted Rand Index (ARI). Table 2 shows that our KC algorithm, which corrects the Gaussian kernel, achieves the best clustering performance under a large missing ratio  $80\%$  (i.e., missing completely at random).
Table 2: Performance of standard spectral clustering on incomplete data under a missing ratio  ${80}\%$  .
Dataset	Metric	Naive	ZERO	MEAN	kNN	EM	SVT	GR	KFMC	DC	TRF	EE	KC
Yale64	ACC	0.561	0.218	0.365	0.227	0.374	0.224	0.218	0.259	0.513	0.562	0.573	0.578
NMI	0.588	0.246	0.429	0.257	0.428	0.269	0.264	0.297	0.551	0.587	0.593	0.596
PUR	0.572	0.233	0.390	0.241	0.396	0.241	0.232	0.273	0.525	0.570	0.581	0.584
ARI	0.353	0.010	0.137	0.015	0.145	0.017	0.012	0.035	0.293	0.350	0.357	0.366
Umist	ACC	0.462	0.220	0.351	0.230	0.349	0.218	0.410	0.314	0.350	0.462	0.461	0.463
NMI	0.669	0.282	0.478	0.314	0.479	0.286	0.597	0.423	0.488	0.667	0.669	0.673
PUR	0.549	0.245	0.415	0.261	0.419	0.247	0.502	0.366	0.408	0.546	0.551	0.553
ARI	0.373	0.070	0.206	0.082	0.207	0.067	0.304	0.140	0.216	0.370	0.371	0.377
USPS	ACC	0.343	0.350	0.362	0.360	0.222	0.353	0.375	0.351	0.418	0.464	0.511	0.523
NMI	0.222	0.228	0.278	0.265	0.104	0.236	0.319	0.358	0.312	0.393	0.457	0.472
PUR	0.395	0.400	0.434	0.427	0.260	0.402	0.440	0.460	0.473	0.535	0.594	0.609
ARI	0.168	0.164	0.180	0.175	0.051	0.167	0.206	0.178	0.231	0.304	0.344	0.360
Isolet	ACC	0.495	0.260	0.245	0.297	0.172	0.263	0.357	0.324	0.243	0.515	0.560	0.561
NMI	0.613	0.339	0.330	0.383	0.228	0.342	0.492	0.465	0.323	0.643	0.660	0.672
PUR	0.520	0.278	0.261	0.318	0.183	0.277	0.377	0.354	0.260	0.542	0.584	0.593
ARI	0.364	0.126	0.108	0.159	0.055	0.131	0.210	0.177	0.105	0.387	0.429	0.432
We also examine robustness across varying missing ratios. Fig. 1 shows the KC method's advantage over imputation, especially at large missingness. Numerical results are in Appendix E.
- The KC method consistently surpasses others in kernel estimation, evidenced by reduced relative errors (RE-K) and enhanced neighborhood ties in the  $k\mathrm{NN}$  graph, seen through higher Recall values.
- At small missing ratios, the KC method offers incremental improvements over imputation. However, at large missing ratios, imputation methods falter due to scant observed data, leading to increased errors and compromised clustering. Conversely, the KC method maintains stable clustering outcomes.
ZERO MEAN kNN EM SVT GR KFMC Kc
Figure 1: Robustness analysis of kernel estimation, neighborhood relationship, and standard spectral clustering on the Yale64 and Umist datasets under a wide range of missing ratios, i.e.,  $r \in [20\%, 80\%]$ .
5.3 Comparative Studies on Self-expressive Affinity Learning
We delve deeper into the advantages of the corrected Gaussian kernel in kernel self-expressive affinity learning, encompassing methods like KSSC [19], KLSR [3], KSL-Sp, and AKLSR, with KSL-Pp excluded due to its impracticality. As Table 3 illustrates, our KC method excels over baselines in the NMI. Notably, both KSL-Sp and AKLSR algorithms employ corrected kernels to yield dependable affinity matrices, thereby elevating performance. Furthermore, our imputation-free framework could augment spectral clustering using other kernels, such as the Exponential kernel. A comprehensive comparison and in-depth analysis are available in Appendix F. Collectively, our study presents a holistic approach to clustering incomplete data, paving the way for subsequent research.
Table 3: NMI performance of self-expressive affinity on incomplete data under a missing ratio  ${80}\%$  .
Method	Dataset	Naive	ZERO	MEAN	kNN	EM	SVT	GR	KFMC	DC	TRF	EE	KC
KSSC	Yale64	0.219	0.215	0.167	0.173	0.177	0.218	0.208	0.259	0.588	0.210	0.209	0.616
Umist	0.101	0.151	0.198	0.154	0.178	0.141	0.639	0.254	0.123	0.121	0.139	0.714
USPS	0.017	0.196	0.396	0.271	0.222	0.200	0.360	0.359	0.077	0.018	0.025	0.410
Isolet	0.065	0.024	0.153	0.384	0.382	0.022	0.454	0.292	0.171	0.064	0.062	0.582
KLSR	Yale64	0.606	0.311	0.604	0.320	0.609	0.321	0.327	0.318	0.597	0.603	0.604	0.616
Umist	0.676	0.579	0.640	0.538	0.638	0.576	0.630	0.548	0.647	0.671	0.687	0.696
USPS	0.019	0.225	0.417	0.317	0.235	0.231	0.356	0.360	0.418	0.019	0.420	0.485
Isolet	0.215	0.398	0.548	0.463	0.400	0.400	0.551	0.512	0.522	0.306	0.628	0.659
KSL-Sp	Yale64	0.370	0.315	0.581	0.303	0.579	0.305	0.304	0.295	0.555	0.364	0.599	0.619
Umist	0.144	0.453	0.595	0.485	0.592	0.454	0.672	0.443	0.616	0.152	0.680	0.690
USPS	0.073	0.288	0.360	0.342	0.196	0.285	0.331	0.439	0.238	0.055	0.517	0.524
Isolet	0.266	0.410	0.556	0.487	0.364	0.410	0.485	0.557	0.383	0.246	0.624	0.634
AKLSR	Yale64	0.452	0.327	0.606	0.338	0.605	0.308	0.338	0.312	0.570	0.464	0.575	0.614
Umist	0.379	0.490	0.624	0.498	0.620	0.484	0.691	0.456	0.611	0.350	0.626	0.691
USPS	0.021	0.114	0.336	0.235	0.236	0.115	0.098	0.017	0.327	0.031	0.259	0.365
Isolet	0.100	0.246	0.448	0.381	0.370	0.246	0.069	0.072	0.453	0.119	0.411	0.496
5.4 Spectral Clustering on Block-missing Data: A Case Study
Beyond the experiments addressing data missing completely at random, we generate a block of appropriate sizes located randomly in images and values in the block are missing where the missingness is systematically related to locations. As illustrated in Table 4, under this block-missing paradigm [70], our KC method persistently outperforms existing data imputation and distance calibration techniques.
Table 4: NMI performance of self-expressive affinity on block-missing data under a missing ratio  $80\%$ . Bold font indicates the best, and underline indicates the second-best.
Dataset	Method	Naive	ZERO	MEAN	kNN	EM	SVT	GR	KFMC	DC	TRF	EE	KC
Yale64	KSSC	0.217	0.340	0.526	0.379	0.530	0.376	0.342	0.496	0.482	0.207	0.212	0.562
KLSR	0.592	0.336	0.544	0.365	0.534	0.365	0.338	0.533	0.484	0.596	0.585	0.592
KSL-Sp	0.488	0.335	0.540	0.367	0.544	0.366	0.343	0.543	0.481	0.484	0.584	0.591
AKLSR	0.548	0.342	0.544	0.367	0.539	0.378	0.342	0.540	0.467	0.546	0.578	0.605
In this paper, we propose an imputation-free framework for spectral clustering on incomplete data. Our framework directly learns high-quality affinity matrices via kernel correction and affinity learning, improving clustering performance. We introduce a novel kernel correction method with a theoretical guarantee, new affinity learning methods, and an adaptive extension that simultaneously learns high-quality kernel and affinity matrices. Extensive experiments demonstrate the effectiveness of our proposed methods on various benchmark datasets, showing superior performance compared to existing data imputation and distance calibration approaches. Our work provides a systematic solution to clustering incomplete data, with future work focused on integrating with deep clustering methods for various types of incomplete data and enhancing the speed and scalability of the algorithms.
We appreciate the anonymous reviewers for their insightful feedback that greatly enhanced this paper. The work of Fangchen Yu and Wenye Li was supported in part by Guangdong Basic and Applied Basic Research Foundation (2021A1515011825), Guangdong Introducing Innovative and Entrepreneurial Teams Fund (2017ZT07X152), Shenzhen Science and Technology Program (CUHKSZWDZC0004), and Shenzhen Research Institute of Big Data Scholarship Program. The work of Jicong Fan was partially supported by the Youth program 62106211 of the National Natural Science Foundation of China. The work of Yicheng Zeng was supported by the Shenzhen Science and Technology Program (Grant No. RCBS20221008093336086), the Youth Program (Grant No. 12301383) of National Natural Science Foundation of China, and the Internal Project Fund from Shenzhen Research Institute of Big Data (Grant No. J00220230012). The work of Jianfeng Mao was supported in part by the National Natural Science Foundation of China (NSFC) under grant U1733102, in part by the National Natural Science Foundation of China (NSFC) under grant 72394362, in part by the Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen) under grant B10120210117, and in part by CUHK-Shenzhen under grant PF.01.000404.
[1] Andrew Ng, Michael Jordan, and Yair Weiss. On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems, 14, 2001.
[2] Ulrike Von Luxburg. A tutorial on spectral clustering. Statistics and Computing, 17:395-416, 2007.
[3] Jicong Fan, Yiheng Tu, Zhao Zhang, Mingbo Zhao, and Hajun Zhang. A simple approach to automated spectral clustering. Advances in Neural Information Processing Systems, 35:9907-9921, 2022.
[4] Jianbo Shi and Jitendra Malik. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8):888-905, 2000.
[5] Stella X Yu and Jianbo Shi. Multiclass spectral clustering. In IEEE International Conference on Computer vision, volume 2, pages 313-313. IEEE Computer Society, 2003.
[6] Víctor Mijangos, Gerardo Sierra, and Azucena Montes. Sentence level matrix representation for document spectral clustering. Pattern Recognition Letters, 85:29-34, 2017.
[7] Tae Jin Park, Kyu J Han, Manoj Kumar, and Shrikanth Narayanan. Auto-tuning spectral clustering for speaker diarization using normalized maximum eigengap. IEEE Signal Processing Letters, 27:381-385, 2019.
[8] Marcilio CP De Souto, Ivan G Costa, Daniel SA de Araujo, Teresa B Ludermir, and Alexander Schliep. Clustering cancer gene expression data: a comparative study. BMC Bioinformatics, 9(1):1-14, 2008.
[9] Seyoung Park and Hongyu Zhao. Spectral clustering based on learning similarity matrix. Bioinformatics, 34(12):2069-2076, 2018.
[10] Nauman Shahid, Vassilis Kalofolias, Xavier Bresson, Michael Bronstein, and Pierre Vanderheynst. Robust principal component analysis on graphs. In Proceedings of the IEEE International Conference on Computer Vision, pages 2812-2820, 2015.
[11] Jicong Fan and Tommy W.S. Chow. Sparse subspace clustering for data with missing entries and high-rank matrix completion. Neural Networks, 93:36-44, 2017.
[12] Jicong Fan. Large-scale subspace clustering via k-factorization. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pages 342-352, 2021.
[13] Jicong Fan and Tommy W. S. Chow. Exactly robust kernel principal component analysis. IEEE Transactions on Neural Networks and Learning Systems, 31(3):749-761, 2020.
This document serves as supplementary material for the paper entitled "Boosting Spectral Clustering on Incomplete Data via Kernel Correction and Affinity Learning". It contains a theoretical analysis with proofs and formula derivation of theorems and algorithms presented in the main paper, a detailed description of experimental settings, and additional numerical results. Specifically,
Sec. A and B provide the proof of Theorem 1 and the formula derivations of Kernel Self-expressive Learning (KSL) algorithms with an adaptive extension, respectively;
Sec. C illustrates experimental settings and implementation details;
Sec. D validates the benefits of the kernel correction algorithm on kernel and distance estimation;
Sec. E and F present comprehensive results and in-depth analysis on clustering performance.
A Theoretical Property of Kernel Correction
Theorem 1.  $\| K^{*} - \hat{K}\|_{F}\leq \| K^{*} - K^{0}\|_{F}$ . The equality holds if and only if  $K^0\succeq 0$ , i.e.,  $K^0 = \hat{K}$