Original: S IMPLE

Original: Y ET

Original:  E FFECTIVE G RAPH C ONTRASTIVE L EARNING FOR R ECOMMENDATION
Revised: L IGHT GCL: S IMPLE Y ET E FFECTIVE G RAPH C ONTRASTIVE L EARNING FOR R ECOMMENDATION

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Graph neural network (GNN) is a powerful learning approach for graph-based recommender systems.
Revised: Graph neural network (GNN) is a powerful learning approach for graph-based recommender systems.

Original: Recently, GNNs integrated with contrastive learning have shown superior performance in recommendation with their data augmentation schemes, aiming at dealing with highly sparse data.
Revised: Recently, GNNs integrated with contrastive learning have shown superior performance in recommendation with their data augmentation schemes, aiming at dealing with highly sparse data.

Original: Despite their success, most existing graph contrastive learning methods either perform stochastic augmentation (e g., node/edge perturbation) on the user-item interaction graph or rely on the heuristic-based augmentation techniques (e g., user clustering) for generating contrastive views.
Revised: Despite their success, most existing graph contrastive learning methods either perform stochastic augmentation (e.g., node/edge perturbation) on the user-item interaction graph, or rely on the heuristic-based augmentation techniques (e.g., user clustering) for generating contrastive views.

Original: We argue that these methods cannot well preserve the intrinsic semantic structures and are easily biased by the noise perturbation.
Revised: We argue that these methods cannot well preserve the intrinsic semantic structures and are easily biased by the noise perturbation.

Original: In this paper, we propose a simple yet effective graph contrastive learning paradigm LightGCL that mitigates these issues impairing the generality and robustness of CL-based recommenders.
Revised: In this paper, we propose a simple yet effective graph contrastive learning paradigm LightGCL that mitigates these issues impairing the generality and robustness of CL-based recommenders.

Original: Our model exclusively utilizes singular value decomposition for contrastive augmentation, which enables the unconstrained structural refinement with global collaborative relation modeling.
Revised: Our model exclusively utilizes singular value decomposition for contrastive augmentation, which enables the unconstrained structural refinement with global collaborative relation modeling.

Original: Experiments conducted on several benchmark datasets demonstrate the significant improvement in performance of our model over the state-of-the-arts.
Revised: Experiments conducted on several benchmark datasets demonstrate the significant improvement in performance of our model over the state-of-the-arts.

Original: Further analyses demonstrate the superiority of LightGCL’s robustness against data sparsity and popularity bias.
Revised: Further analyses demonstrate the superiority of LightGCL’s robustness against data sparsity and popularity bias.

Original: The source code of our model is available at https://anonymous.4open.science/r/LightGCL/.
Revised: The source code of our model is available at https://github.com/HKUDS/LightGCL .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Graph neural networks (GNNs) have shown effectiveness in graph-based recommender systems by extracting local collaborative signals via neighborhood representation aggregation (Wang et al., 2019;
Revised: Graph neural networks (GNNs) have shown effectiveness in graph-based recommender systems by extracting local collaborative signals via neighborhood representation aggregation (Wang et al., 2019;

Original: Chen et al., 2020b).
Revised: Chen et al., 2020b).

Original: In general, to learn user and item representations, GNN-based recommenders perform embedding propagation on the user-item interaction graph by stacking multiple message passing layers for exploring high-order connectivity (He et al., 2020; Zhang et al., 2019;
Revised: In general, to learn user and item representations, GNN-based recommenders perform embedding propagation on the user-item interaction graph by stacking multiple message passing layers for exploring high-order connectivity (He et al., 2020; Zhang et al., 2019;

Original: Liu et al., 2021a).
Revised: Liu et al., 2021a).

Original: Most GNN-based collaborative filtering models adhere to the supervised learning paradigm, requiring sufficient quality labelled data for model training.
Revised: Most GNN-based collaborative filtering models adhere to the supervised learning paradigm, requiring sufficient quality labelled data for model training.

Original: However, many practical recommendation scenarios struggle with the data sparsity issue in learning high-quality user and item representations from limited interaction data (Liu et al., 2021b; Lin et al., 2021).
Revised: However, many practical recommendation scenarios struggle with the data sparsity issue in learning high-quality user and item representations from limited interaction data (Liu et al., 2021b; Lin et al., 2021).

Original: To address the label scarcity issue, the benefits of contrastive learning have been brought into the recommendation for data augmentation (Wu et al., 2021).
Revised: To address the label scarcity issue, the benefits of contrastive learning have been brought into the recommendation for data augmentation (Wu et al., 2021).

Original: The main idea of contrastive learning in enhancing the user and item representation is to research the agreement between the generated embedding views by contrasting the defined positive pairs with negative instance counterparts (Xie et al., 2022).
Revised: The main idea of contrastive learning in enhancing the user and item representation is to research the agreement between the generated embedding views by contrasting the defined positive pairs with negative instance counterparts (Xie et al., 2022).

Original: While contrastive learning has been shown to be effective in improving the performance of graphbased recommendation methods, the view generators serve as the core part of data augmentation through identifying accurate contrasting samples.
Revised: While contrastive learning has been shown to be effective in improving the performance of graphbased recommendation methods, the view generators serve as the core part of data augmentation through identifying accurate contrasting samples.

Original: Most of current graph contrastive learning (GCL) approaches employ heuristic-based contrastive view generators to maximize the mutual information between the input positive pairs and push apart negative instances(Wu et al., 2021;
Revised: Most of current graph contrastive learning (GCL) approaches employ heuristic-based contrastive view generators to maximize the mutual information between the input positive pairs and push apart negative instances(Wu et al., 2021;

Original: Yu et al., 2022a;
Revised: Yu et al., 2022a;

Original: Xia et al., 2022b).
Revised: Xia et al., 2022b).

Original: To construct perturbed views, SGL (Wu et al., 2021) has been proposed to generate node pairs of positive view by corrupting the structural information of user-item interaction graph using stochastic augmentation strategies, e.g., node dropping and edge perturbation.
Revised: To construct perturbed views, SGL (Wu et al., 2021) has been proposed to generate node pairs of positive view by corrupting the structural information of user-item interaction graph using stochastic augmentation strategies, e.g., node dropping and edge perturbation.

Original: To improve the graph contrastive learning in recommendation, SimGCL (Yu et al., 2022a) offers embedding augmentation with random noise perturbation.
Revised: To improve the graph contrastive learning in recommendation, SimGCL (Yu et al., 2022a) offers embedding augmentation with random noise perturbation.

Original: To work on identifying semantic neighbors of nodes (users and items), HCCF (Xia et al., 2022b) and NCL (Lin et al., 2022) are introduced to pursue consistent representations between the structurally adjacent nodes and semantic neighbors.
Revised: To work on identifying semantic neighbors of nodes (users and items), HCCF (Xia et al., 2022b) and NCL (Lin et al., 2022) are introduced to pursue consistent representations between the structurally adjacent nodes and semantic neighbors.

Original: Despite their effectiveness, state-of-the-art contrastive recommender systems suffer from several inherent limitations: i) Graph augmentation with random perturbation may lose useful structural information, which misleads the representation learning.
Revised: Despite their effectiveness, state-of-the-art contrastive recommender systems suffer from several inherent limitations: i) Graph augmentation with random perturbation may lose useful structural information, which misleads the representation learning.

Original: ii)
Revised: ii)

Original: The success of heuristic-guided representation contrasting schemes is largely built upon the view generator, which limits the model generality and is vulnerable to the noisy user behaviors.
Revised: The success of heuristic-guided representation contrasting schemes is largely built upon the view generator, which limits the model generality and is vulnerable to the noisy user behaviors.

Original: iii)
Revised: iii)

Original: Most of current GNN-based contrastive recommenders are limited by the over-smoothing issue which leads to indistinguishable representations.
Revised: Most of current GNN-based contrastive recommenders are limited by the over-smoothing issue which leads to indistinguishable representations.

Original: In light of the above limitations and challenges, we revisit the graph contrastive learning paradigm for recommendation with a proposed simple yet effective augmentation method LightGCL.
Revised: In light of the above limitations and challenges, we revisit the graph contrastive learning paradigm for recommendation with a proposed simple yet effective augmentation method LightGCL.

Original: In our model, the graph augmentation is guided by singular value decomposition (SVD) to not only distill the useful information of user-item interactions but also inject the global collaborative context into the representation alignment of contrastive learning.
Revised: In our model, the graph augmentation is guided by singular value decomposition (SVD) to not only distill the useful information of user-item interactions but also inject the global collaborative context into the representation alignment of contrastive learning.

Original: Instead of generating two handcrafted augmented views, important semantic of user-item interactions can be well preserved with our robust graph contrastive learning paradigm.
Revised: Instead of generating two handcrafted augmented views, important semantic of user-item interactions can be well preserved with our robust graph contrastive learning paradigm.

Original: This enables our self-augmented representations to be reflective of both user-specific preferences and cross-user global dependencies.
Revised: This enables our self-augmented representations to be reflective of both user-specific preferences and cross-user global dependencies.

Original: Our contributions are highlighted as follows:
Revised: Our contributions are highlighted as follows:

Original: • In this paper, we enhance the recommender systems by designing a lightweight and robust graph contrastive learning framework to address the identified key challenges pertaining to this task.
Revised: • In this paper, we enhance the recommender systems by designing a lightweight and robust graph contrastive learning framework to address the identified key challenges pertaining to this task.

Original: • We propose an effective and efficient contrastive learning paradigm LightGCL for graph augmentation.
Revised: • We propose an effective and efficient contrastive learning paradigm LightGCL for graph augmentation.

Original: With the injection of global collaborative relations, our model can mitigate the issues brought by inaccurate contrastive signals.
Revised: With the injection of global collaborative relations, our model can mitigate the issues brought by inaccurate contrastive signals.

Original: • Our method exhibits improved training efficiency compared to existing GCL-based approaches.
Revised: • Our method exhibits improved training efficiency compared to existing GCL-based approaches.

Original: • Extensive experiments on several real-world datasets justify the performance superiority of our LightGCL.
Revised: • Extensive experiments on several real-world datasets justify the performance superiority of our LightGCL.

Original: In-depth analyzes demonstrate the rationality and robustness of LightGCL.
Revised: In-depth analyzes demonstrate the rationality and robustness of LightGCL.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Graph Contrastive Learning for Recommendation .
Revised: Graph Contrastive Learning for Recommendation .

Original: A promising line of recent studies has incorporated contrastive learning (CL) into graph-based recommenders, to address the label sparsity issue with self-supervision signals.
Revised: A promising line of recent studies has incorporated contrastive learning (CL) into graph-based recommenders, to address the label sparsity issue with self-supervision signals.

Original: Particularly, SGL (Wu et al., 2021) and SimGCL (Yu et al., 2022a) perform data augmentation over graph structure and embeddings with random dropout operations.
Revised: Particularly, SGL (Wu et al., 2021) and SimGCL (Yu et al., 2022a) perform data augmentation over graph structure and embeddings with random dropout operations.

Original: However, such stochastic augmentation may drop important information, which may make the sparsity issue of inactive users even worse.
Revised: However, such stochastic augmentation may drop important information, which may make the sparsity issue of inactive users even worse.

Original: Furthermore, some recent alternative CL-based recommenders (HCCF (Xia et al., 2022b) and NCL (Lin et al., 2022)) design heuristic-based strategies to construct view for embedding contrasting.
Revised: Furthermore, some recent alternative CL-based recommenders, such as HCCF (Xia et al., 2022b) and NCL (Lin et al., 2022), design heuristic-based strategies to construct view for embedding contrasting.

Original: Despite their effectiveness, the successof them heavily relies on their incorporated heuristics (e.g., the number of hyperedges or user clusters) for contrastive view generation, which can hardly be adaptive to different recommendation tasks.
Revised: Despite their effectiveness, their success heavily relies on their incorporated heuristics (e.g., the number of hyperedges or user clusters) for contrastive view generation, which can hardly be adaptive to different recommendation tasks.

Original: Self-Supervised Learning on Graphs .
Revised: Self-Supervised Learning on Graphs .

Original: Recently, self-supervised learning (SSL) has advanced the graph learning paradigm to enhance node representation from unlabeled graph data (Zhu et al., 2021a;b; Velickovic et al., 2019; Hassani & Khasahmadi, 2020; Peng et al., 2020; Zhu et al., 2020;
Revised: Recently, self-supervised learning (SSL) has advanced the graph learning paradigm by enhancing node representation from unlabeled graph data (Zhu et al., 2021a;b; Velickovic et al., 2019; Hassani & Khasahmadi, 2020; Peng et al., 2020; Zhu et al., 2020;

Original: Wu et al., 2022).
Revised: Wu et al., 2022).

Original: For example, to improve the predictive SSL paradigm, AutoSSL (Jin et al., 2022) automatically combines multiple pretext tasks for augmentation.
Revised: For example, to improve the predictive SSL paradigm, AutoSSL (Jin et al., 2022) automatically combines multiple pretext tasks for augmentation.

Original: Towards the line of contrastive
Revised: Towards the line of contrastive

Original: SSL over graph structures, recent efforts focus on designing various graph contrastive learning methods (Yu et al., 2022b;
Revised: SSL over graph structures, recent efforts focus on designing various graph contrastive learning methods (Yu et al., 2022b;

Original: Yin et al., 2022; Zhang et al., 2022; Xia et al., 2022a; Suresh et al., 2021).
Revised: Yin et al., 2022; Zhang et al., 2022; Xia et al., 2022a; Suresh et al., 2021).

Original: For instance, SimGRACE Xia et al.
Revised: For instance, SimGRACE Xia et al.

Original: (2022a) proposes to generate contrastive views with the
Revised: (2022a) proposes to generate contrastive views with the

Original: GNN encoder perturbations.
Revised: GNN encoder perturbations.

Original: In AutoGCL Yin et al.
Revised: In AutoGCL Yin et al.

Original: (2022), graph view generators are designed to be jointly trained with the graph encoder in an end-to-end way.
Revised: (2022), graph view generators are designed to be jointly trained with the graph encoder in an end-to-end way.

Original: Additionally, GCA (Zhu et al., 2021b) performs both topology-level and attribute-level data augmentation for contrastive view generation.
Revised: Additionally, GCA (Zhu et al., 2021b) performs both topology-level and attribute-level data augmentation for contrastive view generation.

Original: In this method, important edges and features will be identified for adaptive augmentation.
Revised: In this method, important edges and features will be identified for adaptive augmentation.

Original: GraphCL (You et al., 2020) generates correlated graph representation views using various augmentation strategies, such as node/edge perturbation and attribute masking.
Revised: GraphCL (You et al., 2020) generates correlated graph representation views using various augmentation strategies, such as node/edge perturbation and attribute masking.

Original: Original Graph
Revised: Original Graph

Original: ApproxSVD
Revised: ApproxSVD

Original: [Figure]
Revised: [Figure]

Original: Reconstruct
Revised: Reconstruct

Original: [Figure]
Revised: [Figure]

Original: View View
Revised: View View

Original: GCNs
Revised: GCNs

Original: [Figure]
Revised: [Figure]

Original: Pairwise Loss for Recommendation ℒ (

Original:  M ETHODOLOGY
Revised:  M ETHODOLOGY

Original: In this section, we describe our proposed LightGCL framework in detail.
Revised: In this section, we describe our proposed LightGCL framework in detail.

Original: LightGCL is a lightweight graph contrastive learning paradigm as illustrated in Fig.
Revised: LightGCL is a lightweight graph contrastive learning paradigm as illustrated in Fig.

Original: Complementary to the GCN backbone (the upper half of the figure) extracting the local graph dependency, the SVD-guided augmentation (the lower half of the figure) empowers the graph contrastive learning with global collaborative relation analysis for learning effective user and item representations.
Revised: Complementary to the GCN backbone (the upper half of the figure) extracting the local graph dependency, the SVD-guided augmentation (the lower half of the figure) empowers the graph contrastive learning with global collaborative relation analysis for learning effective user and item representations.

Original: 3. L OCAL G RAPH D EPENDENCY M ODELING
Revised: 3. L OCAL G RAPH D EPENDENCY M ODELING

Original: As a common practice of collaborative filtering, we assign each user u i and item v j with an embedding vector e ( u ) i , e ( v ) j ∈ R d , where d is the embedding size.
Revised: As a common practice of collaborative filtering, we assign each user u i and item v j with an embedding vector e ( u ) i , e ( v ) j ∈ R d , where d is the embedding size.

Original: The collections of all user and item embeddings are defined as E ( u ) ∈ R
Revised: The collections of all user and item embeddings are defined as E ( u ) ∈ R

Original: I × d and E ( v )
Revised: I × d and E ( v )

Original: ∈ R J × d , where I and J are the number of users and items, respectively.
Revised: ∈ R J × d , where I and J are the number of users and items, respectively.

Original: Following Xia et al.
Revised: Following Xia et al.

Original: (2022b)
Revised: (2022b)

Original: , we adopt a two-layer GCN to aggregate the neighboring information for each node.
Revised: , we adopt a two-layer GCN to aggregate the neighboring information for each node.

Original: In layer l , the aggregation process is expressed as follows:
Revised: In layer l , the aggregation process is expressed as follows:

Original: [Equation]
Revised: [Equation]

Original: (1) where z ( u ) i,l and z ( v ) j,l denote the l -th layer aggregated embedding for user u i and item v j . σ ( · ) represents the LeakyReLU with a negative slope of 0.5.
Revised: (1) where z ( u ) i,l and z ( v ) j,l denote the l -th layer aggregated embedding for user u i and item v j . σ ( · ) represents the LeakyReLU with a negative slope of 0.5.

Original: ˜ A is the normalized adjacency matrix, on which we perform the edge dropout denoted as p ( · ) , to mitigate the overfitting issue.
Revised: ˜ A is the normalized adjacency matrix, on which we perform the edge dropout denoted as p ( · ) , to mitigate the overfitting issue.

Original: We implement the residual connections in each layer to retain the original information of the nodes as follows:
Revised: We implement the residual connections in each layer to retain the original information of the nodes as follows:

Original: [Equation]
Revised: [Equation]

Original: The final embedding for a node is the sum of its embeddings across all layers, and the inner product between the final embedding of a user u i and an item v j predicts u
Revised: The final embedding for a node is the sum of its embeddings across all layers, and the inner product between the final embedding of a user u i and an item v j predicts u

Original: i ’s preference towards v j :
Revised: i ’s preference towards v j :

Original: [Equation]
Revised: [Equation]

Original: 3. E FFICIENT G LOBAL C OLLABORATIVE R ELATION L EARNING
Revised: 3. E FFICIENT G LOBAL C OLLABORATIVE R ELATION L EARNING

Original: To empower graph contrastive learning for recommendation with global structure learning, we equip our LightGCL with the SVD scheme (Rajwade et al., 2012; Rangarajan, 2001) to efficiently distill important collaborative signals from the global perspective.
Revised: To empower graph contrastive learning for recommendation with global structure learning, we equip our LightGCL with the SVD scheme (Rajwade et al., 2012; Rangarajan, 2001) to efficiently distill important collaborative signals from the global perspective.

Original: Specifically, we first perform SVD on the adjacency matrix A as A = USV ⊤ .
Revised: Specifically, we first perform SVD on the adjacency matrix A as A = USV ⊤ .

Original: Here, U / V is an I × I /
Revised: Here, U / V is an I × I /

Original: J ×
Revised: J ×

Original: J orthonormal matrix with columns being the eigenvectors of A ’s row-row / column-column correlation matrix.
Revised: J orthonormal matrix with columns being the eigenvectors of A ’s row-row / column-column correlation matrix.

Original: S is an I × J diagonal matrix storing the singular values of A .
Revised: S is an I × J diagonal matrix storing the singular values of A .

Original: The largest singular values are usually associated with the principal components of the matrix.
Revised: The largest singular values are usually associated with the principal components of the matrix.

Original: Thus, we truncate the list of singular values to keep the largest q values, and reconstruct the adjacency matrix with the truncated matrices as ˆ A =
Revised: Thus, we truncate the list of singular values to keep the largest q values, and reconstruct the adjacency matrix with the truncated matrices as ˆ A =

Original: U q S q V ⊤ q , where U q ∈ R
Revised: U q S q V ⊤ q , where U q ∈ R

Original: I × q and V q ∈ R J × q contain the first q columns of U and V respectively.
Revised: I × q and V q ∈ R J × q contain the first q columns of U and V respectively.

Original: S q ∈ R q × q is the diagonal matrix of the q largest singular values.
Revised: S q ∈ R q × q is the diagonal matrix of the q largest singular values.

Original: The reconstructed matrix ˆ A is a low-rank approximation of the adjacency matrix A , for it holds that rank ( ˆ A )
Revised: The reconstructed matrix ˆ A is a low-rank approximation of the adjacency matrix A , for it holds that rank ( ˆ A )

Original: = q .
Revised: = q .

Original: The advantages of SVD-based graph structure learning are two-folds.
Revised: The advantages of SVD-based graph structure learning are two-folds.

Original: Firstly, it emphasizes the principal components of the graph by identifying the user-item interactions that are important and reliable to user preference representations.
Revised: Firstly, it emphasizes the principal components of the graph by identifying the user-item interactions that are important and reliable to user preference representations.

Original: Secondly, the generated new graph structures preserve the global collaborative signals by considering each user-item pair.
Revised: Secondly, the generated new graph structures preserve the global collaborative signals by considering each user-item pair.

Original: Given the ˆ A , we perform message propagation on the reconstructed user-item relation graph in each layer:
Revised: Given the ˆ A , we perform message propagation on the reconstructed user-item relation graph in each layer:

Original: [Equation]
Revised: [Equation]

Original: However, performing the exact SVD on large matrices is highly expensive, making it impractical for handling large-scale user-item matrix.
Revised: However, performing the exact SVD on large matrices is highly expensive, making it impractical for handling large-scale user-item matrix.

Original: Therefore, we adopt the randomized SVD algorithm proposed by Halko et al.
Revised: Therefore, we adopt the randomized SVD algorithm proposed by Halko et al.

Original: , whose key idea is to first approximate the range of the input matrix with a low-rank orthonormal matrix, and then perform SVD on this smaller matrix.
Revised: , whose key idea is to first approximate the range of the input matrix with a low-rank orthonormal matrix, and then perform SVD on this smaller matrix.

Original: [Equation]
Revised: [Equation]

Original: where q is the required rank for the decomposed matrices, and ˆ U q ∈ R
Revised: where q is the required rank for the decomposed matrices, and ˆ U q ∈ R

Original: I × q , ˆ S q ∈ R q × q , ˆ
Revised: I × q , ˆ S q ∈ R q × q , ˆ

Original: V q ∈ R J × q are the approximated versions of U q , S q , V q .
Revised: V q ∈ R J × q are the approximated versions of U q , S q , V q .

Original: Thus, we rewrite the message propagation rules in Eq.
Revised: Thus, we rewrite the message propagation rules in Eq.

Original: 4 with the approximated matrices and the collective representations of the embeddings as follows:
Revised: 4 with the approximated matrices and the collective representations of the embeddings as follows:

Original: [Equation]
Revised: [Equation]

Original: where G ( u ) l and G ( v ) l are the collections of user and item embeddings encoded from the new generated graph structure view.
Revised: where G ( u ) l and G ( v ) l are the collections of user and item embeddings encoded from the new generated graph structure view.

Original: Note that we do not need to compute and store the large dense matrix ˆ
Revised: Note that we do not need to compute and store the large dense matrix ˆ

Original: A SV D .
Revised: A SV D .

Original: Instead, we can store ˆ U q , ˆ S q and ˆ V q , which are of low dimensions.
Revised: Instead, we can store ˆ U q , ˆ S q and ˆ V q , which are of low dimensions.

Original: By pre-calculating ( ˆ U q ˆ S q ) and ( ˆ V q ˆ S q ) during the preprocessing stage with SVD, the model efficiency is improved.
Revised: By pre-calculating ( ˆ U q ˆ S q ) and ( ˆ V q ˆ S q ) during the preprocessing stage with SVD, the model efficiency is improved.

Original: 3. S IMPLIFIED L OCAL -G LOBAL C ONTRASTIVE L EARNING
Revised: 3. S IMPLIFIED L OCAL -G LOBAL C ONTRASTIVE L EARNING

Original: The conventional GCL methods such as SGL and SimGCL contrast node embeddings by constructing two extra views, while the embeddings generated from the original graph (the main-view) are not directly involved in the InfoNCE loss.
Revised: The conventional GCL methods such as SGL and SimGCL contrast node embeddings by constructing two extra views, while the embeddings generated from the original graph (the main-view) are not directly involved in the InfoNCE loss.

Original: The reason for adopting such a cumbersome three-view paradigm may be that the random perturbation used to augment the graph may provide misleading signals to the main-view embeddings.
Revised: The reason for adopting such a cumbersome three-view paradigm may be that the random perturbation used to augment the graph may provide misleading signals to the main-view embeddings.

Original: In our proposed method, however, the augmented graph view is created with global collaborative relations, which can enhance the main-view representations.
Revised: In our proposed method, however, the augmented graph view is created with global collaborative relations, which can enhance the main-view representations.

Original: Therefore, we simplify the CL framework by directly contrasting the SVD-augmented view embeddings g ( u ) i,l with the main-view embeddings z ( u ) i,l in the InfoNCE loss (Oord et al., 2018): 
Revised: Therefore, we simplify the CL framework by directly contrasting the SVD-augmented view embeddings g ( u ) i,l with the main-view embeddings z ( u ) i,l in the InfoNCE loss (Oord et al., 2018): L ( u [Equation] =

Original: [Equation]
Revised: [Equation]

Revised: (cid:88)

Original: where s ( · ) and τ stand for the cosine similarity and the temperature respectively.
Revised: i [Equation] (cid:88) l [Equation]

Revised: −

Revised: [Equation]

Revised: where s ( · ) and τ stand for the cosine similarity and the temperature respectively.

Original: The InfoNCE loss L ( v ) s for the items are defined in the same way.
Revised: The InfoNCE loss L ( v ) s for the items are defined in the same way.

Original: To prevent overfitting, we implement a random node dropout in each batch to exclude some nodes from participating in the contrastive learning.
Revised: To prevent overfitting, we implement a random node dropout in each batch to exclude some nodes from participating in the contrastive learning.

Original: As shown in Eq.
Revised: As shown in Eq.

Original: 8, the contrastive loss is jointly optimized with our main objective function for the recommendation task (
Revised: 8, the contrastive loss is jointly optimized with our main objective function for the recommendation task (

Original: where ˆ y i,p s and ˆ y i,n s denote the predicted scores for a pair of positive and negative items of user i ):
Revised: where ˆ y i,p s and ˆ y i,n s denote the predicted scores for a pair of positive and negative items of user i ):

Original: [Equation]
Revised: [Equation]

Original:  E VALUATION
Revised:  E VALUATION

Original: To verify the superiority and effectiveness of the proposed LightGCL method, we perform extensive experiments to answer the following research questions:
Revised: To verify the superiority and effectiveness of the proposed LightGCL method, we perform extensive experiments to answer the following research questions:

Original: • RQ1 : How does LightGCL perform on different datasets compared to various SOTA baselines?
Revised: • RQ1 : How does LightGCL perform on different datasets compared to various SOTA baselines?

Original: • RQ2 : How does the lightweight graph contrastive learning improve the model efficiency?
Revised: • RQ2 : How does the lightweight graph contrastive learning improve the model efficiency?

Original: • RQ3 : How does our model perform against data sparsity, popularity bias and over-smoothing?
Revised: • RQ3 : How does our model perform against data sparsity, popularity bias and over-smoothing?

Original: • RQ4 : How does the local-global contrastive learning contribute to the performance of our model?
Revised: • RQ4 : How does the local-global contrastive learning contribute to the performance of our model?

Original: • RQ5 : How do different parameter settings affect our model performance?
Revised: • RQ5 : How do different parameter settings affect our model performance?

Original: 4. E XPERIMENTAL S ETTINGS 4.1. D ATASETS AND E VALUATION P ROTOCOLS
Revised: 4. E XPERIMENTAL S ETTINGS 4.1. D ATASETS AND E VALUATION P ROTOCOLS

Original: We evaluate our model and the baselines on five real-world datasets: Yelp (29,601 users, 24, items, 1,517,326 interactions): a dataset collected from the rating interactions on Yelp platform;
Revised: We evaluate our model and the baselines on five real-world datasets: Yelp (29,601 users, 24, items, 1,517,326 interactions): a dataset collected from the rating interactions on Yelp platform;

Original: Gowalla (50,821 users, 57,440 items, 1,069,128 interactions): a dataset containing users’ check-in records collected from Gowalla platform; ML-10M (69,878 users, 10,195 items, 9,988,816 interactions): a well-known movie-rating dataset for collaborative filtering; Amazon-book (78,578 users, 77,801 items, 3,190,224 interactions): a dataset composed of users’ ratings on books collected from
Revised: Gowalla (50,821 users, 57,440 items, 1,172,425 interactions): a dataset containing users’ check-in records collected from Gowalla platform; ML-10M (69,878 users, 10,195 items, 9,988,816 interactions): a well-known movie-rating dataset for collaborative filtering; Amazon-book (78,578 users, 77,801 items, 2,240,156 interactions): a dataset composed of users’ ratings on books collected from

Original: Amazon; and Tmall (47,939 users, 41,390 items, 2,357,450 interactions): a E-commerce dataset containing users’ purchase records on different products in Tmall platform.
Revised: Amazon; and Tmall (47,939 users, 41,390 items, 2,357,450 interactions): a E-commerce dataset containing users’ purchase records on different products in Tmall platform.

Original: In accordance with He et al.
Revised: In accordance with He et al.

Original: (2020) and Wu et al.
Revised: (2020) and Wu et al.

Original: (2021), we split the datasets into training, validation and testing sets with a ratio of 7:2:1.
Revised: (2021), we split the datasets into training, validation and testing sets with a ratio of 7:2:1.

Original: We adopt the Recall@N and Normalized Discounted
Revised: We adopt the Recall@N and Normalized Discounted

Original: Cumulative Gain (NDCG)@N, where N = { 20, 40 } , as the evaluation metrics.
Revised: Cumulative Gain (NDCG)@N, where N = { 20, 40 } , as the evaluation metrics.

Original: 4.1. B ASELINE M ETHODS
Revised: 4.1. B ASELINE M ETHODS

Original: We compare our model against 16 state-of-the-art baselines with different learning paradigms: • MLP-enhanced Collaborative Filtering: NCF (He et al., 2017).
Revised: We compare our model against 16 state-of-the-art baselines with different learning paradigms: • MLP-enhanced Collaborative Filtering: NCF (He et al., 2017).

Original: • GNN-based Collaborative Filtering: GCCF (Chen et al., 2020c), LightGCN (He et al., 2020).
Revised: • GNN-based Collaborative Filtering: GCCF (Chen et al., 2020c), LightGCN (He et al., 2020).

Original: • Disentangled Graph Collaborative Filtering: DGCF (Wang et al., 2020b).
Revised: • Disentangled Graph Collaborative Filtering: DGCF (Wang et al., 2020b).

Original: • Hypergraph-based Collaborative Filtering: HyRec (Wang et al., 2020a).
Revised: • Hypergraph-based Collaborative Filtering: HyRec (Wang et al., 2020a).

Original: • Self-Supervised Learning Recommender Systems:
Revised: • Self-Supervised Learning Recommender Systems:

Original: GraphCL (You et al., 2020), GRACE (Zhu et al., 2020), GCA (Zhu et al., 2021b), MHCN (Yu et al., 2021), SAIL (Yu et al., 2022b), AutoGCL (Yin et al., 2022), SimGRACE (Xia et al., 2022a), SGL (Wu et al., 2021), HCCF (Xia et al., 2022b), SHT (Xia et al., 2022c), SimGCL (Yu et al., 2022a).
Revised: GraphCL (You et al., 2020), GRACE (Zhu et al., 2020), GCA (Zhu et al., 2021b), MHCN (Yu et al., 2021), SAIL (Yu et al., 2022b), AutoGCL (Yin et al., 2022), SimGRACE (Xia et al., 2022a), SGL (Wu et al., 2021), HCCF (Xia et al., 2022b), SHT (Xia et al., 2022c), SimGCL (Yu et al., 2022a).

Original: Due to space limit, the detailed descriptions of baselines are presented in Appendix A.
Revised: Due to space limit, the detailed descriptions of baselines are presented in Appendix A.

Original: 4.1. H YPERPARAMETER S ETTINGS
Revised: 4.1. H YPERPARAMETER S ETTINGS

Original: To ensure a fair comparison, we tune the hyperparameters of all the baselines within the ranges suggested in the original papers, except the following fixed settings for all the models: the embedding size is set as 32; the batch size is 256; two convolutional layers are used for GCN models.
Revised: To ensure a fair comparison, we tune the hyperparameters of all the baselines within the ranges suggested in the original papers, except the following fixed settings for all the models: the embedding size is set as 32; the batch size is 256; two convolutional layers are used for GCN models.

Original: For our LightGCL, the regularization weights λ 1 and λ 2 are tuned from { 1e-5, 1e-6, 1e-7 } and { 1e4, 1e-5 } , respectively.
Revised: For our LightGCL, the regularization weights λ 1 and λ 2 are tuned from { 1e-5, 1e-6, 1e-7 } and { 1e4, 1e-5 } , respectively.

Original: The temperature τ is searched from { 0.3, 0.5, 1, 3 ,10 } .
Revised: The temperature τ is searched from { 0.3, 0.5, 1, 3 ,10 } .

Original: The dropout rate is chosen from { 0, 0.25 } .
Revised: The dropout rate is chosen from { 0, 0.25 } .

Original: The rank (i.e., q ) for SVD, is set as 5.
Revised: The rank (i.e., q ) for SVD, is set as 5.

Original: We use the Adam optimizer with a learning rate of 0.001 decaying at the rate of 0.98 until the rate reaches 0.0005.
Revised: We use the Adam optimizer with a learning rate of 0.001 decaying at the rate of 0.98 until the rate reaches 0.0005.

Original: 4. P ERFORMANCE V ALIDATION (RQ1)
Revised: 4. P ERFORMANCE V ALIDATION (RQ1)

Original: We summarize the experimental result in Table 1 2 , with the following observations and conclusions:
Revised: We summarize the experimental result in Table 1 2 , with the following observations and conclusions:

Original: Due to space limit, results of NCF, GCCF, GraphCL, SAIL, GRACE, and AutoGCL are in Appendix B.
Revised: Due to space limit, results of NCF, GCCF, GraphCL, SAIL, GRACE, and AutoGCL are in Appendix B.

Original: [Table]
Revised: [Table]

Original: • Contrastive Learning Dominates.
Revised: • Contrastive Learning Dominates.

Original: As can be seen from the table, recent methods implementing contrastive learning (SGL, HCCF, SimGCL) exhibit consistent superiority as compared to traditional graph-based (GCCF, LightGCN) or hypergraph-based (HyRec) models.
Revised: As can be seen from the table, recent methods implementing contrastive learning (SGL, HCCF, SimGCL) exhibit consistent superiority as compared to traditional graph-based (GCCF, LightGCN) or hypergraph-based (HyRec) models.

Original: They also perform better than some of other self-supervised learning approaches (MHCN).
Revised: They also perform better than some of other self-supervised learning approaches (MHCN).

Original: This could be attributed to the effectiveness of CL to learn evenly distributed embeddings (Yu et al., 2022a).
Revised: This could be attributed to the effectiveness of CL to learn evenly distributed embeddings (Yu et al., 2022a).

Original: • Contrastive Learning Enhancement.
Revised: • Contrastive Learning Enhancement.

Original: Our method consistently outperforms all the contrastive learning baselines.
Revised: Our method consistently outperforms all the contrastive learning baselines.

Original: We attribute such performance improvement to the effective augmentation of graph contrastive learning via injecting global collaborative contextual signals.
Revised: We attribute such performance improvement to the effective augmentation of graph contrastive learning via injecting global collaborative contextual signals.

Original: Other compared contrastive learning-based recommenders (e.g., SGL, SimGCL, and HCCF) are easily biased by noisy interaction information and generate misleading self-supervised signals.
Revised: Other compared contrastive learning-based recommenders (e.g., SGL, SimGCL, and HCCF) are easily biased by noisy interaction information and generate misleading self-supervised signals.

Original: 4. E FFICIENCY S TUDY (RQ2)
Revised: 4. E FFICIENCY S TUDY (RQ2)

Original: GCL models often suffer from a high computational cost due to the construction of extra views and the convolution operations performed on them during training.
Revised: GCL models often suffer from a high computational cost due to the construction of extra views and the convolution operations performed on them during training.

Original: However, the low-rank nature of the
Revised: However, the low-rank nature of the

Original: SVD-reconstructed graph and the simplified CL structure enable the training of our LightGCL to be highly efficient.
Revised: SVD-reconstructed graph and the simplified CL structure enable the training of our LightGCL to be highly efficient.

Original: We analyze the pre-processing and per-batch training complexity of our model in comparison to three competitive baselines, as summarized in Table 2.
Revised: We analyze the pre-processing and per-batch training complexity of our model in comparison to three competitive baselines, as summarized in Table 2.

Original: [Table]
Revised: [Table]

Original: • Although our model requires performing the SVD in the pre-processing stage which takes O ( qE ) , the computational cost is negligible compared to the training stage since it only needs to be performed once.
Revised: • Although our model requires performing the SVD in the pre-processing stage which takes O ( qE ) , the computational cost is negligible compared to the training stage since it only needs to be performed once.

Original: In fact, by moving the construction of contrastive view to the pre-processing stage, we avoid the repetitive graph augmentation during training, which improves model efficiency.
Revised: In fact, by moving the construction of contrastive view to the pre-processing stage, we avoid the repetitive graph augmentation during training, which improves model efficiency.

Original: • Traditional GCN methods (e.g., LightGCN) only perform convolution on one graph, inducing a complexity of O (2 ELd ) per batch.
Revised: • Traditional GCN methods (e.g., LightGCN) only perform convolution on one graph, inducing a complexity of O (2 ELd ) per batch.

Original: For most GCL-based methods, three contrastive views are
Revised: For most GCL-based methods, three contrastive views are

Original: computed per batch, leading to a complexity of roughly three times of LightGCN.
Revised: computed per batch, leading to a complexity of roughly three times of LightGCN.

Original: Instead, only two contrastive views are involved in our model.
Revised: In our model, instead, only two contrastive views are involved.

Original: Additionally, due to the low-rank property of
Revised: Additionally, due to the low-rank property of

Original: SVD-based graph structure learning, our graph encoder takes only O [2 q ( I + J ) Ld ] time.
Revised: SVD-based graph structure learning, our graph encoder takes only O [2 q ( I + J ) Ld ] time.

Original: For most datasets, including the five we use, 2 q ( I + J ) <
Revised: For most datasets, including the five we use, 2 q ( I + J ) <

Original: E .
Revised: E .

Original: Therefore, the training complexity of our model is less than half of that of the SOTA efficient model SimGCL.
Revised: Therefore, the training complexity of our model is less than half of that of the SOTA efficient model SimGCL.

Original: 4. R ESISTANCE AGAINST D ATA S PARSITY AND P OPULARITY B IAS (RQ3)
Revised: 4. R ESISTANCE AGAINST D ATA S PARSITY AND P OPULARITY B IAS (RQ3)

Original: To evaluate the robustness of our model in alleviating data sparsity, we group the sparse users by their interaction degrees and calculate the Recall@20 of each group on Yelp and Gowalla datasets.
Revised: To evaluate the robustness of our model in alleviating data sparsity, we group the sparse users by their interaction degrees and calculate the Recall@20 of each group on Yelp and Gowalla datasets.

Original: As can be seen from the figures, the performance of HCCF and SimGCL varies across datasets, but our LightGCL consistently outperforms them in all cases.
Revised: As can be seen from the figures, the performance of HCCF and SimGCL varies across datasets, but our LightGCL consistently outperforms them in all cases.

Original: In particular, our model performs notably well on the extremely sparse user group ( < 15 interactions), as the Recall@20 of these users is not much lower (and is even higher on Gowalla ) than that of the whole dataset.
Revised: In particular, our model performs notably well on the extremely sparse user group ( < 15 interactions), as the Recall@20 of these users is not much lower (and is even higher on Gowalla ) than that of the whole dataset.

Original: Additionally, we illustrate our model’s ability to mitigate popularity bias compared to HCCF and
Revised: Additionally, we illustrate our model’s ability to mitigate popularity bias compared to HCCF and

Original: SimGCL.
Revised: SimGCL.

Original: Similar to Section 4.4, we group the long-tail items by their degree of interactions.
Revised: Similar to Section 4.4, we group the long-tail items by their degree of interactions.

Original: Following Wu et al.
Revised: Following Wu et al.

Original: (2021), we adopt the decomposed Recall@20 defined as Recall ( g ) =
Revised: (2021), we adopt the decomposed Recall@20 defined as Recall ( g ) =

Original: | ( V urec ) ( g ) ∩ V utest | | V utest | where V utest refers to the set of test items for the user u , and ( V urec ) ( g ) is the set of Top-K recommended items for u that belong to group g .
Revised: | ( V urec ) ( g ) ∩ V utest | | V utest | where V utest refers to the set of test items for the user u , and ( V urec ) ( g ) is the set of Top-K recommended items for u that belong to group g .

Original: The results are shown in Fig.
Revised: The results are shown in Fig.

Original: Similar to the results on sparse users, HCCF and SimGCL’s performance fluctuates a lot with the influence of popularity bias.
Revised: Similar to the results on sparse users, HCCF and SimGCL’s performance fluctuates a lot with the influence of popularity bias.

Original: Our model performs better in most cases, which shows its resistance against popularity bias.
Revised: Our model performs better in most cases, which shows its resistance against popularity bias.

Original: Note that since the extremely sparse group ( < 15 interactions) is significantly larger than the other groups in Gowalla , they contribute to a large fraction of the Recall@20 , resulting in a different trend from that of Yelp in the figure.
Revised: Note that since the extremely sparse group ( < 15 interactions) is significantly larger than the other groups in Gowalla , they contribute to a large fraction of the Recall@20 , resulting in a different trend from that of Yelp in the figure.

Original: 4. B ALANCING BETWEEN O VER - SMOOTHING AND O VER - UNIFORMITY (RQ3)
Revised: 4. B ALANCING BETWEEN O VER - SMOOTHING AND O VER - UNIFORMITY (RQ3)

Original: In this section, we illustrate the effectiveness of our model in learning a moderately dispersed embedding distribution, by preserving user unique preference pattern and inter-user collaborative dependencies.
Revised: In this section, we illustrate the effectiveness of our model in learning a moderately dispersed embedding distribution, by preserving user unique preference pattern and inter-user collaborative dependencies.

Original: We randomly sample 2,000 nodes from Yelp and Gowalla and map their embeddings to the 2-D space with t-SNE (Van der Maaten & Hinton, 2008).
Revised: We randomly sample 2,000 nodes from Yelp and Gowalla and map their embeddings to the 2-D space with t-SNE (Van der Maaten & Hinton, 2008).

Original: The visualizations of these embeddings are presented in Fig.
Revised: The visualizations of these embeddings are presented in Fig.

Original: We also calculate the Mean Average Distance (MAD) (Chen et al., 2020a) of the embeddings, summarized in Table 3.
Revised: We also calculate the Mean Average Distance (MAD) (Chen et al., 2020a) of the embeddings, summarized in Table 3.

Original: [Table]
Revised: [Table]

Original: As can be seen from Fig.
Revised: As can be seen from Fig.

Original: 4, the embedding distributions of non-CL methods (i.e., LightGCN,
Revised: 4, the embedding distributions of non-CL methods (i.e., LightGCN,

Original: MHCN) exhibit indistinguishable clusters in the embedding space, which indicates the limitation of addressing the over-smoothing issue.
Revised: MHCN) exhibit indistinguishable clusters in the embedding space, which indicates the limitation of addressing the over-smoothing issue.

Original: On the contrary, the existing CL-based methods tend to learn i) over-uniform distributions, e.g., SGL on Yelp learns a huge cloud of evenly-distanced embeddings with no clear community structure to well capture the collaborative relations between users; ii) highly dispersed small clusters with severe over-smoothing issue inside the clusters, e.g., the embeddings of SimGCL on Gowalla appear to be scattered grained clusters inside which embeddings are highly similar.
Revised: On the contrary, the existing CL-based methods tend to learn i) over-uniform distributions, e.g., SGL on Yelp learns a huge cloud of evenly-distanced embeddings with no clear community structure to well capture the collaborative relations between users; ii) highly dispersed small clusters with severe over-smoothing issue inside the clusters, e.g., the embeddings of SimGCL on Gowalla appear to be scattered grained clusters inside which embeddings are highly similar.

Original: Compared with them, clear community structures could be identified by our method to capture collaborative effects, while the embeddings inside each community are reasonably dispersed to be reflective of user-specific preference.
Revised: Compared with them, clear community structures could be identified by our method to capture collaborative effects, while the embeddings inside each community are reasonably dispersed to be reflective of user-specific preference.

Original: The MAD of our model’s learned features is also in between of the two types of baselines as shown in Table 3.
Revised: The MAD of our model’s learned features is also in between of the two types of baselines as shown in Table 3.

Original: 4. A BLATION S TUDY (RQ4)
Revised: 4. A BLATION S TUDY (RQ4)

Original: To investigate the effectiveness of our SVD-based graph augmentation scheme, we perform the ablation study to answer the question of whether we could provide guidance to the contrastive learning with a different approach of matrix decomposition.
Revised: To investigate the effectiveness of our SVD-based graph augmentation scheme, we perform the ablation study to answer the question of whether we could provide guidance to the contrastive learning with a different approach of matrix decomposition.

Original: To this end, we implement two variants of our model, replacing the approximated SVD algorithm with other matrix decomposition methods: CLMF adopts the view generated by a pre-trained MF (Koren et al., 2009); CL-SVD++ utilizes the
Revised: To this end, we implement two variants of our model, replacing the approximated SVD algorithm with other matrix decomposition methods: CLMF adopts the view generated by a pre-trained MF (Koren et al., 2009); CL-SVD++ utilizes the

Original: SVD++ (Koren, 2008) which takes implicit user feedback into consideration.
Revised: SVD++ (Koren, 2008) which takes implicit user feedback into consideration.

Original: As shown in Table 4, with the information distilled from MF or SVD++, the model is able to achieve satisfactory results, indicating the effectiveness of using matrix decomposition to empower CL and the flexibility of our proposed framework.
Revised: As shown in Table 4, with the information distilled from MF or SVD++, the model is able to achieve satisfactory results, indicating the effectiveness of using matrix decomposition to empower CL and the flexibility of our proposed framework.

Original: However, adopting a pre-trained CL component is not only tedious and timeconsuming but also inferior to utilizing the approximate SVD algorithm in terms of performance.
Revised: However, adopting a pre-trained CL component is not only tedious and timeconsuming but also inferior to utilizing the approximate SVD algorithm in terms of performance

Original: [Table]
Revised: [Table]

Original: 4. H YPERPARAMETER A NALYSIS (RQ5)
Revised: 4. H YPERPARAMETER A NALYSIS (RQ5)

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we investigate our model’s sensitivity in relation to several key hyperparameters: the regularization weight for InfoNCE loss λ 1 , the temperature τ
Revised: we investigate our model’s sensitivity in relation to several key hyperparameters: the regularization weight for InfoNCE loss λ 1 , the temperature τ

Original: [Table]
Revised: [Table]

Original: and the required rank of SVD q .
Revised: and the required rank of SVD q .

Original: • The impact of λ 1 .
Revised: • The impact of λ 1 .

Original: As illustrated in Fig.
Revised: As illustrated in Fig.

Original: 6, for the three datasets Yelp , Gowalla and ML-10M , the model’s performance reaches the peak when λ 1 = 10 − 7 .
Revised: 6, for the three datasets Yelp , Gowalla and ML-10M , the model’s performance reaches the peak when λ 1 = 10 − 7 .

Original: It can be noticed that λ 1 with the range of [10 − 6 , 10 − 8 ] can often lead to performance improvement.
Revised: It can be noticed that λ 1 with the range of [10 − 6 , 10 − 8 ] can often lead to performance improvement.

Original: • The impact of τ .
Revised: • The impact of τ .

Original: Fig.
Revised: Fig.

Original: 7 indicates that the model’s performance is relatively stable across different selections of τ from 0 .
Revised: 7 indicates that the model’s performance is relatively stable across different selections of τ from 0 .

Original: 1 to 10 , while the best configuration of τ value varies by datasets.
Revised: 1 to 10 , while the best configuration of τ value varies by datasets.

Original: • The selection of q .
Revised: • The selection of q .

Original: q determines the rank of SVD in our model.
Revised: q determines the rank of SVD in our model.

Original: Experiments have shown that satisfactory results can be achieved with a small q .
Revised: Experiments have shown that satisfactory results can be achieved with a small q .

Original: Specifically, as in Fig.
Revised: Specifically, as in Fig.

Original: 5, we observe that q = 5 is sufficient to preserve important structures of the user-item interaction graph.
Revised: 5, we observe that q = 5 is sufficient to preserve important structures of the user-item interaction graph.

Original: 4. C ASE S TUDY (RQ4)
Revised: 4. C ASE S TUDY (RQ4)

Original: In this section, we present a case study to intuitively show the effectiveness of our model to identify useful knowledge from noisy user-item interactions and make accurate recommendations accordingly.
Revised: In this section, we present a case study to intuitively show the effectiveness of our model to identify useful knowledge from noisy user-item interactions and make accurate recommendations accordingly.

Original: In Fig.
Revised: In Fig.

Original: 8, we can see that the venues visited by user #26 in Yelp mainly fall into two communities: Cleveland (where the user probably lives) and Arizona (where the user may have travelled to).
Revised: 8, we can see that the venues visited by user #26 in Yelp mainly fall into two communities: Cleveland (where the user probably lives) and Arizona (where the user may have travelled to).

Original: In the reconstructed graph, these venues are assigned a new weight according to their potential importance.
Revised: In the reconstructed graph, these venues are assigned a new weight according to their potential importance.

Original: Note that item #2583, a car rental agency in Arizona, has been assigned a negative weight, which conforms to our common sense that people generally would not visit multiple car rental agencies in one trip.
Revised: Note that item #2583, a car rental agency in Arizona, has been assigned a negative weight, which conforms to our common sense that people generally would not visit multiple car rental agencies in one trip.

Original: The SVD-augmented view also provides predictions on invisible links by assigning a large weight 4 to potential venues of interest, such as #2647 and #658.
Revised: The SVD-augmented view also provides predictions on invisible links by assigning a large weight 4 to potential venues of interest, such as #2647 and #658.

Original: Note that when exploiting the graph, the augmented view does not overlook the smaller Arizona community, which enables the model to predict items of minor interests that are usually overshadowed by the majority.
Revised: Note that when exploiting the graph, the augmented view does not overlook the smaller Arizona community, which enables the model to predict items of minor interests that are usually overshadowed by the majority.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we propose a simple and effective augmentation method to the graph contrastive learning framework for recommendation.
Revised: In this paper, we propose a simple and effective augmentation method to the graph contrastive learning framework for recommendation.

Original: Specifically, we explore the key idea of making the singular value decomposition powerful enough to augment user-item interaction graph structures.
Revised: Specifically, we explore the key idea of making the singular value decomposition powerful enough to augment user-item interaction graph structures.

Original: Our key findings indicate that our graph augmentation scheme exhibits strong ability in resisting data sparsity and popularity bias.
Revised: Our key findings indicate that our graph augmentation scheme exhibits strong ability in resisting data sparsity and popularity bias.

Original: Extensive experiments show that our model achieves new state-of-the-art results on several public evaluation datasets.
Revised: Extensive experiments show that our model achieves new state-of-the-art results on several public evaluation datasets.

Original: In future work, we plan to explore the potential of incorporating casual analysis into our lightweight graph contrastive learning model to enhance the recommender system with mitigating confounding effects for data augmentation.
Revised: In future work, we plan to explore the potential of incorporating casual analysis into our lightweight graph contrastive learning model to enhance the recommender system with mitigating confounding effects for data augmentation.

Original: A D ETAILS OF THE B ASELINES
Revised: A D ETAILS OF THE B ASELINES

Original: MLP-enhanced Collaborative Filtering:
Revised: MLP enhanced Collaborative Filtering:

Original: • NCF
Revised: • NCF

Original: (He et al., 2017) is a collaborative filtering model that leverages neural network to exploit non-linearity.
Revised: (He et al., 2017) is a collaborative filtering model that leverages neural network to exploit non-linearity.

Original: Two hidden layers are used in our evaluation.
Revised: Two hidden layers are used in our evaluation.

Original: GNN-based Collaborative Filtering
Revised: GNN-based Collaborative Filtering

Original: : • GCCF (Chen et al., 2020c) strengthens the GNN-based collaborative filtering by implementing a residual network and reducing the non-linear transformation.
Revised: : • GCCF (Chen et al., 2020c) strengthens the GNN-based collaborative filtering by implementing a residual network and reducing the non-linear transformation.

Original: • LightGCN (He et al., 2020) adopts a simplified GCN structure without embedding weight matrices and non-linear projection.
Revised: • LightGCN (He et al., 2020) adopts a simplified GCN structure without embedding weight matrices and non-linear projection.

Original: Disentangled Graph Collaborative Filtering:
Revised: Disentangled Graph Collaborative Filtering:

Original: • DGCF (Wang et al., 2020b) learns a more sophisticated representation by segmenting the embedding vectors to represent multiple latent intentions.
Revised: • DGCF (Wang et al., 2020b) learns a more sophisticated representation by segmenting the embedding vectors to represent multiple latent intentions.

Original: Hypergraph-based Collaborative Filtering:
Revised: Hypergraph -based Collaborative Filtering:

Original: • HyRec (Wang et al., 2020a) makes use of hypergraph to encode multi-order information between users and items.
Revised: • HyRec (Wang et al., 2020a) makes use of hypergraph to encode multi-order information between users and items.

Original: Self-Supervised Learning Recommender Systems:
Revised: Self -Supervised Learning Recommender Systems

Original: • GraphCL
Revised: • GraphCL

Original: (You et al., 2020) utilizes random node dropping and edge masking to generate two contrastive views, which were aligned by optimizing the SSL loss function.
Revised: (You et al., 2020) utilizes random node dropping and edge masking to generate two contrastive views, which were aligned by optimizing the SSL loss function.

Original: • GRACE (Zhu et al., 2020) proposes to corrupt the graph structure by both random edge dropout and random node feature dropping, and uses the corrupted graphs as the contrastive views.
Revised: • GRACE (Zhu et al., 2020) proposes to corrupt the graph structure by both random edge dropout and random node feature dropping, and uses the corrupted graphs as the contrastive views.

Original: • GCA (Zhu et al., 2021b) adaptively dropout the nodes and edges by their importance calculated with node centrality.
Revised: • GCA (Zhu et al., 2021b) adaptively dropout the nodes and edges by their importance calculated with node centrality.

Original: • MHCN (Yu et al., 2021) creates self-supervised signals for the graph representation learning by graph infomax network.
Revised: • MHCN (Yu et al., 2021) creates self-supervised signals for the graph representation learning by graph infomax network.

Original: • SAIL
Revised: • SAIL

Original: (Yu et al., 2022b) maximizes the neighborhood predicting probability between GNNgenerated high-level features and input node features.
Revised: (Yu et al., 2022b) maximizes the neighborhood predicting probability between GNNgenerated high-level features and input node features.

Original: • AutoGCL
Revised: • AutoGCL

Original: (Yin et al., 2022) uses GNN to learn to mask nodes and edges in the augmented graph.
Revised: (Yin et al., 2022) uses GNN to learn to mask nodes and edges in the augmented graph.

Original: It minimizes the similarity between the augmented and the original graph, while maximizing the similarity of the embeddings generated through them, so as to uncover the most important information in the graph.
Revised: It minimizes the similarity between the augmented and the original graph, while maximizing the similarity of the embeddings generated through them, so as to uncover the most important information in the graph.

Original: • SimGRACE (Xia et al., 2022a) creates augmented view by randomly perturbing the parameters of the GNN network.
Revised: • SimGRACE (Xia et al., 2022a) creates augmented view by randomly perturbing the parameters of the GNN network.

Original: • SGL (Wu et al., 2021) adopts random walk sampling and probabilistic edge/node dropout to create augmented views for contrastive learning.
Revised: • SGL (Wu et al., 2021) adopts random walk sampling and probabilistic edge/node dropout to create augmented views for contrastive learning.

Original: In our experiments, we adopt the SGL-ED variant, which implements random edge dropout and exhibits the strongest performance according to the original paper.
Revised: In our experiments, we adopt the SGL-ED variant, which implements random edge dropout and exhibits the strongest performance according to the original paper.

Original: • HCCF (Xia et al., 2022b)
Revised: • HCCF (Xia et al., 2022b)

Original: encodes global graph information with hypergraph and contrasts it against the local information encoded with GCN.
Revised: encodes global graph information with hypergraph and contrasts it against the local information encoded with GCN.

Original: In our experiments, the number of hyper-edges are set as 128 following the original paper.
Revised: In our experiments, the number of hyper-edges are set as 128 following the original paper.

Original: • SHT (Xia et al., 2022c) adopts a hypergraph transformer framework to exploit global collaborative relationships and distills the global information to generate the cross-view self-supervised signals.
Revised: • SHT (Xia et al., 2022c) adopts a hypergraph transformer framework to exploit global collaborative relationships and distills the global information to generate the cross-view self-supervised signals.

Original: In our experiments, the number of hyper-edges are set as 128 following the original paper.
Revised: In our experiments, the number of hyper-edges are set as 128 following the original paper.

Original: • SimGCL
Revised: • SimGCL

Original: (Yu et al., 2022a) propose to simplify the graph augmentation process of contrastive learning by directly injecting random noises into the feature representation.
Revised: (Yu et al., 2022a) propose to simplify the graph augmentation process of contrastive learning by directly injecting random noises into the feature representation.

Original: B P ERFORMANCE C OMPARISON WITH B ASELINES (C ONTINUED )
Revised: B P ERFORMANCE C OMPARISON WITH B ASELINES (C ONTINUED )

Original: In this appendix, we show the performance of NCF, GCCF, GraphCL, SAIL, GRACE
Revised: In this appendix, we show the performance of NCF, GCCF, GraphCL, SAIL, GRACE

Original: [Table]
Revised: [Table]

Original: and AutoGCL, which are not shown in Table 1 due to space limit.
Revised: and AutoGCL, which are not shown in Table 1 due to space limit.

Original: The results are summarized in Table 5.
Revised: The results are summarized in Table 5.

Original: As can be seen from the table, our model outperforms these baselines consistently.
Revised: As can be seen from the table, our model outperforms these baselines consistently.

Original: [Table]
Revised: [Table]

Original: C T HEORETICAL A NALYSIS
Revised: C T HEORETICAL A NALYSIS

Original: We conduct theoretical analyses to show that our local-global CL (Eq7) is augmented to maximize the similarity between embeddings of potentially related nodes, based on the SVD-based global relation learning.
Revised: We conduct theoretical analyses to show that our local-global CL (Eq7) is augmented to maximize the similarity between embeddings of potentially related nodes, based on the SVD-based global relation learning.

Original: Specifically, for a node v j ∈ U , where U = { u i ′ |A
Revised: Specifically, for a node v j ∈ U , where U = { u i ′ |A

Original: i,i ′ =
Revised: i,i ′ =

Original: 0 , ˆ A i,i ′ ̸
Revised: 0 , ˆ A i,i ′ ̸

Original: = 0 } , the embeddings are not updated by s ( z i,l , g i,l ) in the vanilla InfoNCE loss, as v j is not adjacent to u i .
Revised: = 0 } , the embeddings are not updated by s ( z i,l , g i,l ) in the vanilla InfoNCE loss, as v j is not adjacent to u i .

Original: Instead
Revised: Instead

Original: [Table]
Revised: [Table]

Original: our local-global contrastive assigns the following gradients to the embeddings of v j :
Revised: our local-global contrastive assigns the following gradients to the embeddings of v j :

Original: [Equation]
Revised: [Equation]

Original: z i,l , g i,l ) /∂
Revised: i,l , g i,l ) /∂

Original: g i,l − 1 = ∂s [Equation] =
Revised: g i,l − 1 = ∂s 

Original: [Equation]
Revised: [Equation]

Original: (9) where α i,j denotes the normalization weight for node u i and v j .
Revised: (9) where α i,j denotes the normalization weight for node u i and v j .

Original: In this way, the embeddings of nodes in U are also pulled close to s i,l , which injects relatedness information learned by the SVD into the local-global CL optimization.
Revised: In this way, the embeddings of nodes in U are also pulled close to s i,l , which injects relatedness information learned by the SVD into the local-global CL optimization.

Original: D C ALCULATION OF C OMPLEXITY D. A DJACENCY M ATRIX N ORMALIZATION
Revised: D C ALCULATION OF C OMPLEXITY D. A DJACENCY M ATRIX N ORMALIZATION

Original: For a sparse user-item matrix stored in the Coordinate Format (COO), it requires visiting every nonzero elements in the matrix to perform normalization.
Revised: For a sparse user-item matrix stored in the Coordinate Format (COO), it requires visiting every nonzero elements in the matrix to perform normalization.

Original: Thus, the computational complexity is in the order of the number of edges O ( E ) .
Revised: Thus, the computational complexity is in the order of the number of edges O ( E ) .

Original: Note that for the baseline SGL, it requires normalizing the two augmented graph structures during the training phase, each of which contains ρE edges, so it induces a complexity of O (2 ρE ) per batch.
Revised: Note that for the baseline SGL, it requires normalizing the two augmented graph structures during the training phase, each of which contains ρE edges, so it induces a complexity of O (2 ρE ) per batch.

Original: D. A PPROXIMATE SVD A LGORITHM
Revised: D. A PPROXIMATE SVD A LGORITHM

Original: We refer the readers to Halko et al.
Revised: We refer the readers to Halko et al.

Original: (2011) in which the complexity of the approximate SVD algorithm is explained in detail.
Revised: (2011) in which the complexity of the approximate SVD algorithm is explained in detail.

Original: D. G RAPH C ONVOLUTION
Revised: D. G RAPH C ONVOLUTION

Original: Given a sparse COO matrix A with E edges and a dense matrix E with dimensions
Revised: Given a sparse COO matrix A with E edges and a dense matrix E with dimensions

Original: I ( J ) ×
Revised: I ( J ) ×

Original: d , it takes O ( Ed ) time to calculate A E .
Revised: d , it takes O ( Ed ) time to calculate A E .

Original: To perform graph convolution on a graph, we need to multiply the sparse adjacency matrix with E ( v ) l − 1 ∈ R J × d and its transpose with E ( u ) l − 1 ∈ R I × d , which takes
Revised: To perform graph convolution on a graph, we need to multiply the sparse adjacency matrix with E ( v ) l − 1 ∈ R J × d and its transpose with E ( u ) l − 1 ∈ R I × d , which takes

Original: O ( Ed ) each, and O (2 Ed ) in total.
Revised: O ( Ed ) each, and O (2 Ed ) in total.

Original: For L layers, O (2 ELd ) is required.
Revised: For L layers, O (2 ELd ) is required.

Original: For traditional CL-based methods such as SGL and SimCGL, a three-view structure is adopted, resulting in a complexity of
Revised: For traditional CL-based methods such as SGL and SimCGL, a three-view structure is adopted, resulting in a complexity of

Original: O (12 ELd ) (for SGL it again varies a bit depending on ρ ).
Revised: O (12 ELd ) (for SGL it again varies a bit depending on ρ ).

Original: For the SVD-view of our model, ˆ V ⊤ q E ( v ) l − 1 takes O ( qJd ) , and multiplying the result with the precalculated ( ˆ U q ˆ S q ) takes O ( qId ) ; ˆ U ⊤ q E ( v ) l − 1 takes O ( qId ) , and multiplying the result with the precalculated ( ˆ V q ˆ S q ) takes O ( qJd ) .
Revised: For the SVD-view of our model, ˆ V ⊤ q E ( v ) l − 1 takes O ( qJd ) , and multiplying the result with the precalculated ( ˆ U q ˆ S q ) takes O ( qId ) ; ˆ U ⊤ q E ( v ) l − 1 takes O ( qId ) , and multiplying the result with the precalculated ( ˆ V q ˆ S q ) takes O ( qJd ) .

Original: So in total it takes O (2 q ( I + J ) d ) .
Revised: So in total it takes O (2 q ( I + J ) d ) .

Original: D. BPR L OSS
Revised: D. BPR L OSS

Original: In each batch with B users, calculating the scores for positive and negative items both take O ( Bd ) , so in total it takes O (2 Bd ) .
Revised: In each batch with B users, calculating the scores for positive and negative items both take O ( Bd ) , so in total it takes O (2 Bd ) .

Original: D. CL L OSS
Revised: D. CL L OSS

Original: In each batch with B users, calculating the numerator of InfoNCE loss takes O ( Bd ) , and calculating the denominator takes O ( BMd ) where M denotes the total number of nodes in the batch.
Revised: In each batch with B users, calculating the numerator of InfoNCE loss takes O ( Bd ) , and calculating the denominator takes O ( BMd ) where M denotes the total number of nodes in the batch.

Original: Since our model adopts a per layer InfoNCE loss, a factor of L is appended.
Revised: Since our model adopts a per layer InfoNCE loss, a factor of L is appended.