SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Let  $\mathbf{X} = \mathbf{U}\boldsymbol{\Lambda}\mathbf{U}^{\top}$  be an eigenvalue decomposition of  $\mathbf{X}$ , and  $\boldsymbol{\Lambda} = \mathrm{diag}([\lambda_1,\dots,\lambda_d])$ . Choosing  $\phi_{ld}(\mathbf{X}) = -\log \det \mathbf{X} = -\sum_{i}\log \lambda_{i}$  leads to the LogDet divergence  $D_{\phi_{ld}}(\mathbf{X},\mathbf{Y}) = \operatorname {tr}\left(\mathbf{X}\mathbf{Y}^{-1}\right) - \log \det \left(\mathbf{X}\mathbf{Y}^{-1}\right) - d$  where  $\mathbf{X},\mathbf{Y}\in S_{++}^{d}$ . Then generalizing  $\mathcal{L}_{soft}$  in Equation (9) to its LogDet divergence variant by computing  $D_{\phi_{ld}}(\pmb {\Sigma}_{\mathcal{U}},\mathbf{I}) + D_{\phi_{ld}}(\pmb {\Sigma}_{\mathcal{I}},\mathbf{I})$  and removing  $-2d$  constant yields:
$$
\mathcal {L} _ {\text {l o g d e t}} = \operatorname {t r} \left(\boldsymbol {\Sigma} _ {\mathcal {U}} + \boldsymbol {\Sigma} _ {\mathcal {I}}\right) - \log \det  \left(\boldsymbol {\Sigma} _ {\mathcal {U}} \boldsymbol {\Sigma} _ {\mathcal {I}}\right) - 2 d. \tag {12}
$$
Our LogDet decorrelation-enhanced GCF objective becomes:
$$
\mathcal {L} = \mathbb {E} _ {(i, u) \sim \mathcal {R}} \left[ \| f _ {\theta} (u) - f _ {\theta} (i) \| _ {2} ^ {2} \right] + \lambda \cdot \left(\operatorname {t r} \left(\boldsymbol {\Sigma} _ {\mathcal {U}} + \boldsymbol {\Sigma} _ {\mathcal {I}}\right) - \log \det  \left(\boldsymbol {\Sigma} _ {\mathcal {U}} \boldsymbol {\Sigma} _ {\mathcal {I}}\right)\right). \tag {13}
$$
In contrast to the Frobenius-based  $\mathcal{L}_{soft}$ , the LogDet-based penalty  $\mathcal{L}_{\text{logdet}}$  maintains the range space of the covariance matrices and upper bounds the condition number of the matrices. This ensures the user/item embedding matrices are of full rank which prevents the dimensional collapse.
5 Theoretical Analysis.
To use the LogDet divergence based penalty  $\mathcal{L}_{\mathrm{logdet}}$  in GCF, we discuss important to our work theoretical properties of the LogDet divergence and  $\mathcal{L}_{\mathrm{logdet}}$  for rank-deficient matrices.
LogDet Divergence and  $\mathcal{L}_{\text{logdet}}$  for Rank-deficient Matrices.
Corollary 1. LogDet divergence  $D_{\phi_{ld}}(\mathbf{X}, \mathbf{Y})$  is finite iff  $\mathrm{rank}(\mathbf{X}) = \mathrm{rank}(\mathbf{Y})$  [23], which implies that  $\mathcal{L}_{\text{logdet}} = D_{\phi_{ld}}(\pmb{\Sigma}_{\mathcal{U}}, \mathbf{I}) + D_{\phi_{ld}}(\pmb{\Sigma}_{\mathcal{I}}, \mathbf{I}) = \infty$  if  $\mathrm{rank}(\pmb{\Sigma}_{\mathcal{U}}) < \mathrm{rank}(\mathbf{I}) = d \lor \mathrm{rank}(\pmb{\Sigma}_{\mathcal{I}}) < \mathrm{rank}(\mathbf{I}) = d$ .
Corollary 2. Let  $\Sigma_{\mathcal{U}}, \Sigma_{\mathcal{I}} \in S_{++}^{d}$ . Note that  $\mathcal{L}_{soft} = D_{\phi_F}(\Sigma_{\mathcal{U}}, \mathbf{I}) + D_{\phi_F}(\Sigma_{\mathcal{I}}, \mathbf{I})$  is finite if both  $\|\Sigma_{\mathcal{U}}\|_2$  and  $\|\Sigma_{\mathcal{I}}\|_2$  are finite. There exist cases where some infinitesimal change  $\Delta\Sigma_{\mathcal{U}}$  and/or  $\Delta\Sigma_{\mathcal{I}}$  will make  $\Sigma_{\mathcal{U}} + \Delta\Sigma_{\mathcal{U}}$  and/or  $\Sigma_{\mathcal{I}} + \Delta\Sigma_{\mathcal{I}}$  positive semi-definite (PSD). Define the gain of the Bregman divergence based decorrelation penalty as  $g_{\phi}(\mathcal{L}_{reg}) = D_{\phi}(\Sigma_{\mathcal{U}}, \mathbf{I}) + D_{\phi}(\Sigma_{\mathcal{I}}, \mathbf{I}) - [D_{\phi}(\Sigma_{\mathcal{U}} + \Delta\Sigma_{\mathcal{U}}, \mathbf{I}) + D_{\phi}(\Sigma_{\mathcal{I}} + \Delta\Sigma_{\mathcal{I}}, \mathbf{I})]$ . Notice, in case of change from PD to SPD matrix, gain  $g_{\phi}(\mathcal{L}_{reg}) < 0$ . Let the gain of the alignment loss be  $g(\mathcal{L}_{align}) \ll \infty$ . Then one may easily construct examples where  $g(\mathcal{L}_{align}) - g_{\phi_F}(\mathcal{L}_{soft}) > 0$ . That is to say, the net loss  $g(\mathcal{L}_{soft})$  in case of covariance degradation to SPD is lesser than the net gain  $g(\mathcal{L}_{align})$  due to increased user-item alignment. In contrast, from Corollary 1,  $g(\mathcal{L}_{align}) - g_{\phi_{id}}(\mathcal{L}_{logdet}) = -\infty$  in case of covariance degradation to SPD, which prevents the dimensional collapse.
Proof of Corollary 2 is in Appendix A.3.
Corollary 3. Expression - log det  $(\Sigma_{\mathcal{U}}\Sigma_{\mathcal{I}})$  from Equation (12) might fail to yield  $\infty$  if  $\Sigma_{\mathcal{U}}\Sigma_{\mathcal{I}}$  is indefinite. However, covariance matrices are at least SPD by definition, and so is their matrix product.
Corollaries 1, 2 and 3 demonstrate that  $\mathcal{L}_{\mathrm{logdet}}$  naturally maintains the full-rank constraints on covariance matrices of user and item embeddings.
Minimizing  $D_{\phi_{ld}}(\Sigma, \mathbf{I})$  Reduces the Condition Number of  $\Sigma$ . Feature decorrelation is closely related to the condition number. If the spectrum of a matrix is dominated by the leading eigenvalue, the matrix is ill-conditioned and has a large condition number. Let  $\{\lambda_i\}_{i=0}^d$  be the eigenvalues of the matrix  $\Sigma$ . The condition number is defined as  $\operatorname{cond}(\Sigma) = \frac{\lambda \max}{\lambda_{\min}}$ . The ratio between the largest eigenvalue and the smallest eigenvalue quantifies the flatness (isotropy) of the spectrum. For example,  $\operatorname{cond}(\Sigma) = 1$  implies a fully balanced spectrum with  $\lambda_i = \lambda_j$  for all  $i \neq j$  (e.g., spectrum balancing [21, 22, 48, 52]). Below we show that our proposed method minimizes the upper bound of  $\operatorname{cond}(\Sigma)$ .
Lemma 3. The condition number of  $\mathbf{\Sigma}$ ,  $\mathrm{cond}(\mathbf{\Sigma}) = \frac{\lambda_{\max}}{\lambda_{\min}}$ , is upper bounded by  $D_{\phi_{ld}}(\mathbf{\Sigma},\mathbf{I})$  as:
$$
\operatorname {c o n d} (\boldsymbol {\Sigma}) \leq 4 \exp \left(D _ {\phi_ {l d}} (\boldsymbol {\Sigma}, \mathbf {I})\right). \tag {14}
$$
Proof of Lemma 3 is in Appendix A.4.
Lemma 3 implies that the sum of condition numbers of  $\Sigma_{\mathcal{U}}$  and  $\Sigma_{\mathcal{I}}$  is upper bounded by the LogDet divergence based penalty  $\mathcal{L}_{\text{logdet}}$ . Minimizing that penalty implies the flattening of spectrum implies decorrelation/redundancy reduction of embedding.
Datasets. To evaluate the effectiveness of our GCF with the LogDet divergence based penalty  $\mathcal{L}_{\text {logdet }}$ , denoted as  $\mathrm{GCF}_{\text {logdet }}$ , we conduct experiments on three large-scale public datasets: Yelp2018 [15], Amazon-kindle [43], and Alibaba-iFashion [43]. See statistics of these datasets in Appendix A.6. The datasets were split into the training, validation, and test set, with the ratio of  $7:1:2$ . As per the methodology suggested by [43, 15], we first identified the best hyperparameters on the validation set and then merged the training and validation sets to train the model. Finally, the test set was used to evaluate the model using the relevancy-based metric Recall@20 and the ranking-aware metric NDCG@20.
Baselines. Apart from the typical GCF models, i.e., NGCF [41] and LightGCN [15], we also compare  $\mathrm{GCF}_{\text{logdet}}$  with recent data CL-based recommendation models. SGL [43] and DNN+SSL [45] adopt CL as an auxiliary task and conduct feature masking for contrastive learning (CL). BUIR
Table 2: Performance comparison for different contrastive GCF model that optimize uniformity.
Methods	Yelp2018	Amazon-Kindle	Alibaba-iFashion
Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
1 layer	LightGCN	0.0590	0.0484	0.1871	0.1186	0.0845	0.0390
SGL-ED	0.0637	0.0526	0.1936	0.1231	0.0932	0.0442
SimGCL	0.0689	0.0572	0.2087	0.1361	0.1036	0.0505
DirectAU	0.0674	0.0522	0.1971	0.1239	0.0969	0.0426
\( GCF_{logdet} \)	0.0694	0.0577	0.2103	0.1321	0.1060	0.0510
2 layers	LightGCN	0.0622	0.0504	0.2033	0.1284	0.1053	0.0505
SGL-ED	0.0668	0.0549	0.2084	0.1341	0.1062	0.0514
SimGCL	0.0719	0.0601	0.2071	0.1341	0.1119	0.0548
DirectAU	0.0702	0.0584	0.2034	0.1352	0.1043	0.0549
\( GCF_{logdet} \)	0.0720	0.0607	0.2143	0.1381	0.1180	0.0557
3 layers	LightGCN	0.0639	0.0525	0.2057	0.1315	0.0955	0.0461
SGL-ED	0.0675	0.0555	0.2090	0.1352	0.1093	0.0531
SimGCL	0.0721	0.0601	0.2104	0.1374	0.1151	0.0567
DirectAU	0.0713	0.0602	0.2047	0.1385	0.1136	0.0586
\( GCF_{logdet} \)	0.0725	0.0612	0.2151	0.1418	0.1210	0.0601
Table 3: Performance comparison with other models.
Method	Yelp2018	Amazon-Kindle	Alibaba-iFashion
Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
NGCF	0.0579	0.0477	0.1954	0.1263	0.1043	0.0486
LightGCN	0.0639	0.0525	0.2057	0.1315	0.1053	0.0505
NCL	0.0670	0.0562	0.2090	0.1348	0.1088	0.0528
BUIR	0.0487	0.0404	0.0922	0.0528	0.0830	0.0384
DNN+SSL	0.0483	0.0382	0.1520	0.0989	0.0818	0.0375
MixGCF	0.0713	0.0589	0.2098	0.1355	0.1085	0.0520
SGL-ED	0.0675	0.0555	0.2090	0.1352	0.1093	0.0531
SimGCL	0.0721	0.0601	0.2104	0.1374	0.1151	0.0567
DirectAU	0.0713	0.0602	0.2047	0.1385	0.1136	0.0586
\( GCF_{logdet} \)	0.0725	0.0612	0.2151	0.1418	0.1210	0.0601
[24] has a two-branch architecture with a target network and an online network, utilizing only positive examples for self-supervised recommendation. MixGCL [18] introduces the hop mixing technique for synthesizing hard negatives for GCF through embedding interpolation. NCL [27] is a recent contrastive model that designs a prototypical contrastive objective to capture the correlations between a user/item and its context. SimGCL [46] is a CL-based model for recommendation that does not rely on graph augmentation. DirectAU [38] proposes optimizing the alignment and uniformity of both user and item directly for recommendation.
Implementation. We referred to the best hyperparameter settings reported in the original papers of the baselines and then fine-tuned them with the grid search. Detailed settings are in Appendix A.5.
6.1 Main Results
**Decorrelation vs. Uniformity.** Within this section, we compare the performance of our proposed method,  $\mathrm{GCF}_{\text{logdet}}$ , with contrastive-based methods (i.e., SGL, DirectAU) that optimize uniformity to prevent the dimensional collapse in CF. Furthermore, we demonstrate the effectiveness of our method by varying the number of encoder layers. Table 3 shows the performance of different baseline CF methods and our  $\mathrm{GCF}_{\text{logdet}}$ . In the vast majority of cases, contrastive-based methods (e.g., SGL, DirectAU, SimGCL) significantly outperform LightGCN. The largest improvements are observed on Alibaba-iFashion, where the performance of  $\mathrm{GCF}_{\text{logdet}}$  surpasses that of LightGCN under 1 and 3 layers settings. This observation supports our conclusion that LightGCN is susceptible to the dimensional collapse, leading to poor results. Additionally, we observe that DirectAU outperforms SGL, indicating that graph augmentation is not the primary factor for enhancing the performance of the CL-based recommendation model. Instead, our findings demonstrate the importance of designing appropriate loss functions over sophisticated augmentation. Finally,  $\mathrm{GCF}_{\text{logdet}}$  achieves the best results on the all three datasets. Specifically,  $\mathrm{GCF}_{\text{logdet}}$  outperforms DirectAU and SGL, highlighting that decorrelation is more effective than optimizing uniformity for preventing dimensional collapse.
(a) Yelp2018
(b) Kindle
(c) iFashion
Figure 4: The effective rank (higher is better) w.r.t. epoch number and condition number in Yelp2018.
(d) Yelp2018
Table 4: Performance comparison with other models.
Method	Yelp2018	Amazon-Kindle	iFashion
Encoder	Loss	Recall@20	NDCG@20	Recall@20	NDCG@20	Recall@20	NDCG@20
LightGCN	Luni	0.0713	0.0602	0.2107	0.1385	0.1136	0.0586
Lsoft	0.0709	0.0592	0.1982	0.1351	0.1112	0.0580
Stiefel	0.0703	0.0582	0.1882	0.1352	0.1071	0.0528
MLP	Llogdet	0.0725	0.0612	0.2151	0.1418	0.1210	0.0601
Luni	0.0702	0.0589	0.1960	0.1325	0.1085	0.0580
Lsoft	0.0699	0.0582	0.2060	0.0989	0.1018	0.0595
Stiefel	0.0691	0.0579	0.1852	0.1191	0.0958	0.0523
MLP	Llogdet	0.0721	0.0601	0.2154	0.1414	0.1151	0.0607
Performance Comparison with the State of the Art. To further demonstrate the exceptional performance of  $\mathrm{GCF}_{\text {logdet }}$ , we conduct a comparative analysis with several recently proposed recommendation models that are based on augmentation or contrastive learning techniques. As illustrated in Table 3,  $\mathrm{GCF}_{\text {logdet }}$  outperforms the other models by a significant margin, achieving the best performances, respectively. Additionally, NCL and MixGCF, which utilize LightGCN as their backbone, exhibit competitive performance. In contrast, DNN+SSL and BUIR fall short of expectations and are not comparable to LightGCN. We attribute their suboptimal performance to the fact that DNNs have been proven to be effective only when abundant user/item features are available. In our datasets, however, such features are unavailable, and self-supervision signals are created by masking item embeddings, which makes it difficult for these models to perform well in this context. We also report performance on Recall@K and NDCG@K with  $K = 5$ , 10 in Table 9.
6.2 Quantitative Analysis
Measuring the Dimensional Collapse. The evidence of dimensional collapse was identified in contrastive learning by observing the spectrum of representations [19]. The rank of the matrix corresponds to the number of dimensions retained by the transformation (i.e., the dimension of its range) but reveals nothing about the shape of the spectrum. The effective rank [33] (see the definition in the Appendix A.7) quantifies how balanced the spectrum is by means of the spectral entropy. Thus, to show the dimensional collapse tendency when training the model, instead of showing the distribution of eigenvalues, we use the effective rank. Figure 4 shows that the effective rank increases as the training progresses, indicating an increasingly balanced eigenvalue distribution.  $\mathrm{GCF}_{\text{logdet}}$  consistently achieves higher effective rank compared to other methods. This finding not only verifies the existence of the dimensional collapse in CF but also indicates the effectiveness of our  $\mathrm{GCF}_{\text{logdet}}$ . Figure 4d further shows that the condition number decreases as more batches are processed, thus balancing the spectrum of embedding matrices.
Overcoming the Popularity Bias. Below we investigate whether our  $\mathrm{GCF}_{\text{logdet}}$  can reduce the popularity bias by promoting more uniform representations and alleviating the problem of dimensional collapse. We partition the test set into three subsets based on item popularity. Specifically, we label  $80\%$  of the items with the fewest clicks/purchases as "Unpopular,"  $5\%$  of the most clicked/purchased items as "Popular," and the remaining items as "Normal." We then conduct experiments to measure Recall@20 contributed by each group, with the overall Recall@20 value being the sum of the values
Figure 5: LogDet vs. other baselines. LightGCN is the backbone used in all methods.
Model	Yelp2018	iFasion
Unpop.	Norm.	Pop.	Unpop.	Norm.	Pop.
SGL	0.63	0.83	-0.11	0.33	0.05	-0.08
SimGCL	0.52	1.21	0.16	0.44	0.14	-0.03
DirectAU	0.33	0.91	0.11	0.27	0.07	-0.07
\( GCF_{soft} \)	0.41	1.10	0.13	0.41	0.03	-0.01
\( GCF_{logdet} \)	1.51	1.48	0.25	0.94	0.18	0.05
Table 5: The numerical relative improvement of recall@20 in Figure 5 over LightGCN.
from all three groups. The results are presented in Figure 5 and Table 5, which show that our method significantly improves the ability to recommend unpopular items. In contrast, LightGCN tends to recommend popular items and achieves the highest recall value on Normal and Popular items.
Ablations on Different Decorrelation Penalties. Table 4 compares the performance on LightGCN and MLP equipped with different decorrelation penalties.  $\mathcal{L}_{\text {logdet }}$  appears to consistently outperform other penalties, thus verifying its ability to deal with the dimensional collapse and balancing the spectrum of embeddings. Appendix A.8 compares the KL Matrix Divergence and the Riemannian metric in terms of speed. Appendix A.9 compares different distance types.
Graph Contrastive Learning applies CL to the graph domain. By adapting DeepInfoMax [1] to graph representation learning, DGI [37] learns embedding by maximizing the mutual information to discriminate between nodes of original and corrupted graphs. REFINE [54] uses a simple negative sampling term inspired by skip-gram models. COLES [57] and GLEN [56] link the GCL to the form of Laplacian Eigenmap with negative sampling, extended also to image domain by EASE [55]. GRACE [59]/GraphCL [13] create views via graph augmentation for node/graph-level task. COSTA [51] and SFA [52] create graph views via feature augmentations.
Graph Collaborative Filtering. Graph Neural Networks (GNNs) have proven to be powerful architectures for modeling recommendation data, replacing MLP-based models and improving the performance of neural recommender systems [20, 15]. GNNs are a foundation for several state-of-the-art recommendation models. The most common GNN variant is GCN, which has driven the development of graph neural recommendation models, such as GCMC, NGCF, and LightGCN [3, 41, 15]. These GCN-based models refine embeddings and perform graph reasoning by gathering neighborhood information in the user-item graph. Among such models, LightGCN stands out for its effectiveness and simplicity by eliminating transformation matrices and non-linear activation functions. The success of LightGCN has inspired subsequent CL-based recommendation models, such as SGL and SimGCL [43, 46].
Graph Contrastive Learning for Recommendation. Inspired by the success of CL in other fields [11, 5, 35], CL has been combined with recommendation systems. Yao et al. [45] proposed a feature dropout based two-tower architecture for large-scale item recommendation. NCL [27] designed a prototypical contrastive objective to capture the correlations between a user/item and its context. SimGCL [46] adds noise to the embedding space to perform feature augmentation to enhance the performance of recommendations. DirectAU [38] proposes optimizing the alignment and uniformity of both user and item during training. The most widely used model, SGL [43], applies edge/node dropout to augment the graph data. Although these methods have demonstrated their effectiveness, they pay little attention to why CL can enhance recommendations.
We have investigated the popularity bias in Graph Collaborative Filtering, and showed it results from the dimensional collapse of the embedding space. We have demonstrated that the commonly used backbone in GCF, LightGCN, is prone to pose the dimensional collapse. To this end, we have proposed the LogDet divergence based decorrelation penalty for the GCF. We have showed that  $\mathrm{GCF}_{\text {logdet }}$  promotes full-rank embedding matrices and yields small condition number with a theoretical guarantee. For these reasons,  $\mathrm{GCF}_{\text {logdet }}$  has outperformed other contrastive-based GCF models on several benchmark datasets and improved the performance for unpopular items.
The work described in this paper was partially supported by the National Key Research and Development Program of China (No. 2018AAA0100204), RGC Research Impact Fund (RIF), R5034-18 (CUHK 2410021), and RGC General Research Funding Scheme (GRF) 14222922 (CUHK 2151185). PK is supported by CSIRO's Science Digital.
[1] P. Bachman, R. D. Hjelm, and W. Buchwalter. Learning representations by maximizing mutual information across views. NeurIPS, pages 15509-15519, 2019.
[2] A. Bardes, J. Ponce, and Y. LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. arXiv preprint arXiv:2105.04906, 2021.
[3] R. v. d. Berg, T. N. Kipf, and M. Welling. Graph convolutional matrix completion. arXiv preprint arXiv:1706.02263, 2017.
[4] J. Chen, H. Dong, X. Wang, F. Feng, M. Wang, and X. He. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems, 41(3):1-39, 2023.
[5] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597-1607. PMLR, 2020.
[6] Y. Chen, Y. Fang, Y. Zhang, and I. King. Bipartite graph convolutional hashing for effective and efficient top-n search in hamming space. In Proceedings of the ACM Web Conference 2023 (WebConf). ACM, 2023.
[7] Y. Chen, M. Yang, Y. Zhang, M. Zhao, Z. Meng, J. Hao, and I. King. Modeling scale-free graphs with hyperbolic geometry for knowledge-aware recommendation. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining (WSDM), pages 94-102, 2022.
[8] Y. Chen, Y. Yang, Y. Wang, J. Bai, X. Song, and I. King. Attentive knowledge-aware graph convolutional networks with collaborative guidance for personalized recommendation. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), pages 299–311. IEEE, 2022.
[9] Y. Chen, Y. Zhang, M. Yang, Z. Song, C. Ma, and I. King. Wsfe: Wasserstein sub-graph feature encoder for effective user segmentation in collaborative filtering. The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2023.
[10] A. Cherian, S. Sra, A. Banerjee, and N. Papanikolopoulos. Jensen-bregman logdet divergence with application to efficient similarity search for covariance matrices. IEEE transactions on pattern analysis and machine intelligence, 35(9):2161-2174, 2012.
[11] T. Gao, X. Yao, and D. Chen. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821, 2021.
[12] G. H. Golub and C. F. Van Loan. Matrix computations. JHU press, 2013.
[13] H. Hafidi, M. Ghogho, P. Ciblat, and A. Swami. Graphcl: Contrastive self-supervised learning of graph representations. arXiv preprint arXiv:2007.08025, 2020.
[14] W. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[15] X. He, K. Deng, X. Wang, Y. Li, Y. Zhang, and M. Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 639–648, 2020.
[16] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S. Chua. Neural collaborative filtering. In WWW, pages 173-182. International World Wide Web Conferences Steering Committee, 2017.
[54] H. Zhu and P. Koniusz. REFINE: Random range finder for network embedding. In ACM Conference on Information and Knowledge Management, 2021.
[55] H. Zhu and P. Koniusz. EASE: Unsupervised discriminant subspace learning for transductive few-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9078-9088, June 2022.
[56] H. Zhu and P. Koniusz. Generalized laplacian eigenmaps. Advances in Neural Information Processing Systems, 35:30783-30797, 2022.
[57] H. Zhu, K. Sun, and P. Koniusz. Contrastive laplacian eigenmaps. Advances in Neural Information Processing Systems, 34:5682-5695, 2021.
[58] M. Zhu, X. Wang, C. Shi, H. Ji, and P. Cui. Interpreting and unifying graph neural networks with an optimization framework. In Proceedings of the Web Conference 2021, pages 1215-1226, 2021.
[59] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Deep graph contrastive representation learning. 2020.
Mitigating the Popularity Bias of Graph Collaborative
Filtering: A Dimensional Collapse Perspective
(Supplementary Material)
Yifei Zhang†, Hao Zhu‡, Yankai Chen†, Zixing Song†, Piotr Koniusz*,‡, Irwin King†
$^{\ddagger}$ Australian National University,  ${}^{\S}$ Data61♥CSIRO
A Appendices
Proof 1 One can set derivative of Equation (3) with respect to  $\mathbf{Z}$  to zero and get the optimal  $\mathbf{Z}$  as:
$$
\frac {\partial \left[ \| \mathbf {Z} - \tilde {\mathbf {A}} \mathbf {E} \| _ {F} ^ {2} + \eta \operatorname {t r} \left(\mathbf {Z} ^ {T} \mathbf {L} \mathbf {Z}\right) \right]}{\partial \mathbf {Z}} = 0 \quad \Rightarrow \quad \mathbf {Z} - \tilde {\mathbf {A}} \mathbf {E} + \eta \mathbf {L} \mathbf {Z} = 0. \tag {15}
$$
Note that  $\operatorname{det}(\mathbf{I} + \eta \mathbf{L}) > 0$ , thus matrix  $\{\mathbf{I} + \eta \mathbf{L}\}^{-1}$  exists. Then the corresponding closed-form solution can be written as: