Contrastive Graph Structure Learning via Information Bottleneck for Recommendation
Authors: Chunyu Wei
Jian Liang
Di Liu
Fei Wang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Recommender system
Graph Neural Networks
contrastive learning

Abstract: Graph convolution networks (GCNs) for recommendations have emerged as an important research topic due to their ability to exploit higher-order neighbors. Despite their success, most of them suffer from the popularity bias brought by a small number of active users and popular items. Also, a real-world user-item bipartite graph contains many noisy interactions, which may hamper the sensitive GCNs. Graph contrastive learning show promising performance for solving the above challenges in recommender systems. Most existing works typically perform graph augmentation to create multiple views of the original graph by randomly dropping edges/nodes or relying on predefined rules, and these augmented views always serve as an auxiliary task by maximizing their correspondence. However, we argue that the graph structures generated from these vanilla approaches may be suboptimal, and maximizing their correspondence will force the representation to capture information irrelevant for the recommendation task. Here, we propose a Contrastive Graph Structure Learning via Information Bottleneck (CGI) for recommendation, which adaptively learns whether to drop an edge or node to obtain optimized graph structures in an end-to-end manner. Moreover, we innovatively introduce the Information Bottleneck into the contrastive learning process to avoid capturing irrelevant information among different views and help enrich the final representation for recommendation. Extensive experiments on public datasets are provided to show that our model significantly outperforms strong baselines.

Contrastive Graph Structure Learning via Information Bottleneck for Recommendation
Chunyu Wei'; Jian Liang', Di Liu', Fei Fei Wang"
'Alibaba Group. China
Department of Population Health Sciences, Weill Cornell Medicine, USA
weicy15@icloud com
(xuelang .jj, wendi 1i1 @al wwwbabbacannn.... com few2001 @med. cornell..... edu
Abstract
Graph convolution networks (GCNs) for recommendations have emerged as an important research topic due to their ability exploit higher-order neighbors. Despite their success, most of them suffer from the popularity bias brought by small number of active users and popular items. Also, real-world user-item bipartite graph contains many noisy interactions, which may hamper the sensitive GCNs. Graph contrastive learning show promising performance for solving the above challenges in recommender systems. Most existing works typically perform graph augmentation create multiple views of the original graph by randomly dropping edges/nodes or relying on predefined rules, and these augmented views always serve as an auxiliary task by maximizing their correspondence. However, we argue that the graph structures generated from these vanilla approaches may suboptimal, and maximizing their correspondence will force the representation to capture information irrelevant for the recommendation task. Here, we propose a Contrastive Graph Structure Learning via Information Bottleneck (CGI) for recommendation, which adaptively learns whether to drop edge or node to obtain optimized graph structures in an end-to-end manner. Morcover, we innovatively introduce the Information Bottleneck into the contrastive learning process to avoid capturing irrelevant information among different views and help enrich the final representation for recommendation. Extensive experiments public datasets provided to show that our model significantly outperforms strong baselines. g
1 Introduction
Recommender systems have been widely deployed alleviate information overload in diverse scenarios including e-commerce, online news and multimedia contents, which requires high-quality user and item representations learned from the historical interactions [7 [3). Recently, thanks to the powerful capability modeling graph-structured data, Graph Convolution Networks (GCNs) provide efficient way integrate multi-hop neighbors into node representation learning and show prominent performance in recommendation [37
g
Although encouraging performance has been achieved, argue that most GCN-based recommender models suffer from the following two limitations, of which the impacts on the user's exhibited preference presented in Fig. [ i) Popularity Bias. Items inherently have different customer sizes, and this imbalance can potentially lead to popularity bias (45). In most recommender systems, the customer size for items usually follows long-tai tail distribution, which means few items have massive customers while the majority have few customers. Similarly, most users have few interactions. This skewed data distribution will bias GCN-base models towards the popular users and items easily
"qqual contributions from both authors. This work is done when Chunyu Wei works as an intern at Alibaba. The code is available on https///ithub.cm//wieyyy
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

during multi-hop convolution, which may hamper the representation learning. ii) Interaction Noises. User-item interactions usually contain noises especially in the scenarios with only implicit feedbacks (e.g., clicks and purchases). More specifically, these noisy edges in the bipartite graph are not
necessarily aligned with user preferences [8] since it's common that the user clicks something by mistake or finds something boring after purchasing. GCN-based models are known to be vulnerable to
the quality of the input graphs [44]. which means aggregating mislcading neighborhood information is likely to lead to sub-optimal performance. Recent advances in graph contrastive learning 127 (38] have identified an effective training scheme for mitigating popularity bias and increasing robustness for noise graph- graph-based tasks, which inspire many studies [31 41 33
to introduce this training
scheme to enhance representation learning for recommendations.
Nevertheless, existing studies have two limitations. First, most methods perform data augmentation by randomly dropping edges/nodes to change the graph structure [31], shuffling the embeddings to corrupt the node representations [41], or relying pre- defined rules [6]. However, within unsupervised settings, structures created from these vanilla ap- proaches may be suboptimal for recommendation tasks and also lack persuasive rationales for why the randomly dropped edges/nodes alleviate the popu- larity bias and interaction noises. Like the obtained representation N in Fig. [ structures created from these vanilla approaches may deviate from the optimal area. Second, most methods generate multiple views only to serve as an auxiliary task by maximizing the agreement of node representations among these views, which may force the user or item representation in different views to capture the information irrelevant for the recommendation task. For example, the obtained representation No.1 in Fig. D contains much information irrelevant to the real preference. So we believe that a good augmentation (e.g., No.2 in Fig. D should cover as much optimal area as possible while being as small as possible to reduce useless information.
User's Real Preference
User's E Exhibited Preference
O0 ptimaal User Represenlation
000 User" Interacted items
Noisy Interaction
Popular Item
Figure 1: A possible illustration of some user's interactions and preference. Dotted circles de- note possible augmentation representations.
To address the aforementioned limitations, we propose Contrastive Graph Structure Learning via Information Bottleneck (CGI) for recommendation, which contains two key components: learnable graph augmentation and information bottleneck contrastive learning.
First, we propose learnable graph augmentation to learn whether to drop edge or node to transform the original bipartite graph into correlated views, which will be jointly optimized with the downstream recommendation in an end-to-end fashion. As result, these generated views can intentionally reduce the influence of popular nodes while preserving information of the isolated nodes, and thus help to mitigate the popularity bias. The intuition behind is that random dropout will indiscriminately drop nodes or edges regardless of the corresponding node degrees, while by message passing mechanism, GCNs are easier to reconstruct the missing information of popular users or items, but much harder to reconstruct those isolated nodes with few connections, thus may overemphasize those high-degree nodes. These generated views with debiased information are all fed into the GCN-based recommender for multi-view representation learning to increase the ability against popularity bias.
Second, we proposed to integrate different views into a compact representation for the downstream recommendation tasks, which can further improve the robustness of the model. Generally, when information from different views complements each other, it can be expected that the multi-view representation learning approaches can improve downstream performance [28). So we argue that simply maximizing the mutual information in the conventional graph contrastive learning may push the representations of different views to capture information irrelevant to the downstream task. Inspired by the recent advances of Information Bottleneck (IB) (32), which encourages the representation to capture the minimum sufficient information for the downstream task, we utilize the IB principle to minimize the mutual information between the original graph and the generated views while maintaining the downstream recommendation performance of each view. By doing so, the learnable graph augmenters can learn to remove noisy interactions in the original graph much as possible, since these interactions are of no help for the downstream recommendation. Also, the IB principle helps representations of different views to capture collaborative information of different semantics complement to each other.

---

The contributions of this paper are summarized as follows. (1) We propose the CGI to construct optimized graph structures by dropping nodes and edges adaptively for the multi-view representation learning of users and items, which provides rationales for alleviating the popularity bias. (2) To efficiently drop information irrelevant to the downstream recommendation, we innovatively integrate information bottleneck into the multi-view contrastive learning process for recommendation and prove that it can better mitigate interaction noises. (3) Experimental results show that our method outperforms the late-of-the-art methods on three benchmark datasets from different domains.
2 Related Work
Graph-based Recommendation Early works exploiting the user-item bipartite graph for recom- mendation like ItemRank 3 usually followed the label propagation mechanism to propagate users' preference over the graph, i.e., encouraging connected nodes to have similar labels. In recent years, Graph Convolution Networks (GCNs) have made great progress in representation learning tasks including node classification and link prediction [ 12 35 Motivated by the strength of GCNs, several works [24 S 37 30 have adapted GCNs on the user-item bipartite graph to learn more robust
latent representations for users and items in recommender systems.
Contrastive Learning Contrastive Learning (CL) 22 9 was firstly proposed to train CNNs for image representation learning. Graph Contrastive Learning (GCL) applies the idea of CL on GNNs. DGI [27] and InfoGraph [19] learn node representations according to the mutual information between nodes and the whole graph. Peng et al. [15] developed an unsupervised learning model trained by maximizing mutual information of nodes between the input and output of a graph neural encoder. Hu et al. [10] extend the idea to build contrastive pairs between nodes and subgraphs. In addition, GCC [6 designs the pre-training task as subgraph instance discrimination in and across networks and leverage CL to empower GNNs. And a very recent work SGL (31) supplements the classical supervised task of recommendation with an auxiliary graph CL task, which generates multiple views of a node and maximizes the agreement between different views. However, it differs from our work in: (1) SGL [31] generates contrastive pairs by randomly dropping edges/nodes, while our work adopts learnable augmenter to optimize the generated views. (2) SGL (3I) utilizes conventional CL as an auxiliary task by maximizing the agreement of augmentation views, while we propose to encourage the differences between the augmentation views and the original graph.
Information Bottleneck (IB) [23 is an approach based on information theory, which states that if the obtained representation discards information from the input which is not useful for given task, it will increase robustness for the downstream tasks. Besides, the information bottleneck principle is used in multi-view representation learning [34 29 2 Formally,
Learning by CntommtioonBottlenee
given the original data X with label Y. IB is to obtain compact and effective representation Z of X. And the objective of the IB principle is as follows:
max (YY 2) 3I(x: Z). Z
(1)
where 3 is the coefficient to balance the mutual information / ((Y, Z) and 1 (X,
Recently, some works proposed to integrate the IB principle into the graph learning process. You et al. [9 propose a variational graph auto-encoder to generate contrastive views and the downstream contrastive learning utilizes IB performing on graph representations as the unsupervised loss. Both Yu et al. [40] and Yu et al. 42 aim to directly reveal the vital substructure in the subgraph level, among which [1] learns node assignment matrix to extract the subgraph, and implements the IB of two graphs by estimating the KL-divergence from graph latent representation with a statistic network (DONSKER-VARADHAR) Representation of Kl--diergencce. And Yu et al. 42 employ noise injection to manipulate the graph, and customizes the Gaussian prior for each input graph and the injected noise, so to implement the IB of two graphs with tractable variational upper bound. Our CGI differs from them, since we do not directly aim to find an optimal graph structure, instead we try to learn the graph structure complementing the original one. Then by integrating different views into compact representation, we obtain the optimal node representation for the downstream task. Sun et al. 20 learn to mask node feature and generates new structure with the masked feature. Afterward, 20 adopt GNN to learn the distribution of graph representation and utilize the KL-divergence between the learned distribution and the prior distribution to implement the IB.
All these methods aim to find a better structure or representation to replace the original graph for the downstream task, while our CGI follows multi-view representation learning schema. IB is
3

---

utilized to minimize the mutual information between the original graph and the generated views while maintaining the downstream recommendation performance of each view. Besides the noise-invariance property, IB helps representations of different views to capture collaborative information of different semantics that complement each other.
AD-GCL [21] shares some ideas with our CGI but there are fundamental differences. Specifically, AD-GCL focuses on training self-supervised GNNs for graph-level tasks. In contrast, CGI aims to mitigate the popularity bias and interaction noises of node-level collaborative filtering (CF). In addition, AD-GCL adopts an adversarial strategy aiming to maximize the agreement of final representations of different views. Instead, our CGI minimizes the mutual information of different views to capture collaborative information of different semantics. To the best of our knowledge, this is the first study on leveraging the IB principle to enhance graph-based recommendations.
3 Preliminaries
Problem Definition. Let U = [wi, U2.... Um denotes the set of users, and let I {úr, i2, 222....,f,, 3n denotes the set of items. We typically use a binary matrix R E Rm to store user-item interactions (e.g.. purchases and clicks), where Tus = 1 indicates that user u consumed item i while rus = 0 means that item i is unexposed to user works [30, [8. we represent interaction data as a user-item bipartite graph G = {V.E), where the node set V = U U I and the edge set & = {eualras 1, u E u, E I). The adjacency matrix Ag can be formulated as follows:
is not interested in item i. Following most existing
0 Rd
R 0
Ag =
(2)
With respect to the adjacency matrix Ag. the degree matrix D3 E [(m+t)x(m+n)) is diagonal matrix, in which each entry Doli, i] denotes the number of nonzero entries in the i-th row of Ag.
GCN Paradigm. The core of graph convolution on graph G is to update the ego node by aggregating the representations of its neighbor nodes, which can be formulated as follows:
E GCN(E "
where E(I-1) is the current representations of nodes and E is the updated representations after the graph convolution layer. E(0) is the initial inputs, which are usually the ID embeddings (trainable parameters). From the vector level, Eq.
]ann be interpreted as:
(
f(t)
combine (c((-1), f() JJgggreaae (fo.1) E NNN)),
(4)
=
f(l Sombine (e)
(e(--)) f(0))
aggreggat (fe ju E N.))),
(5)
where N₂ and N₁ are the neighbor node set of user u and item respectively. There are many works designing different feombine and faggregate [5] 26 B5}. Usually, there will be readout function to generate the final representations for the recommendation task:
e = freadous (fe 1 0, O.
(6)
For example, freadout can be concatenation 300, weighted sum [8] and retaining the last output 24
LightGCN Brief. In this paper, we implement our CGI on the simple but effective GCN-based recommendation model LightGCN. It adopts weighted sum aggregators and abandon the of feature transformation and nonlinear activation, of which the matrix form can be formulated as:
E (D5 + A + """")) EIEN
where 1(-
- E 1 the output of the previous LightGCN layer or the initial E(O), At last, LightGCN implement the freadout by weighted sum, in which the weight of each layer is set as T.+T following the original work.
After obtaining the representations of users and items, the inner product A. = e e is used to predict preference score, which is commonly adopted in most recommender system: LightGCN employ the Bayesian Personalized Ranking (BPR) loss [17] to optimize the model parameters: Cree = E(us)re (fno(fus fuz), where O = {u, (u, E Rt, (u, R is the pairwise training data, in which R+ denotes the observed interactions, and R- denotes the unobserved interactions. In this work, we also choose it as the objective function for the recommendation task.

---

GCN Layer
Reparametriation
ND
END
I(E, Eyp)
Node Embedklngg
GCN Layer
E
Recommending Loss
Original Bipartite Graph
Emboddings
Minimize
MLP
I(E, Egn
EED
Reparametrizalion
JED
GCN Layer
E
User
Layer
Layer
Embeddings
Figure 2: The overview the CGI framework. We integrate both the node-dropping and edge- views together for a more comprehensive representation, though they can be applied separately.
Methodology
The framework of CGI is illustrated in Fig. 2 and we detail the inference in Appendix.
4.1 Learnable Multi-View Augmentation
Most of GCN-based recommendation like LighTGCN fully relies on the adjacency matrix Ag to refine the representations of users and items in Eq. However. Ag may contain many biased and noisy information as discussed in Sec. [ which continue to propagate misleading information as the LightGCN goes deeper. On the other hand, the vanilla randomly dropout in most contrastive learning for recommendation cannot create powerful views alleviate popularity bias and interaction noises. We hence utilize parameterized networks to generate the layer-wise optimized augmentation views. Specifically, we assign different graph convolution layers with different learned subgraphs coupled with the downstream recommendation and thus obtain multi-view user and item representations. We elaborate on two types of learnable augmentations as follows.
Pooe-Dropping View As illustrated in Sect. [ popular users items in the graph may skew the data distribution and thus hinder the GCN-based recommender. So we perform learnable node dropping at each layer mask those the influential nodes and create the Node- bode-Dropping view, which can be formulated as:
S (1r ④ p.2 v E V),
where p. E {0, 1} is drawn from a Bernoulli distribution parameterized by w i.c. P. (ern(w())) which denotes whether to keep the node U,.
Simply removing the selected node alongside all its connections will cause a dramatic change of the bipartite graph structure thus exerting influence the information aggregation and making the training unstable. Thus instead of removing the selected node, we replace the selected node e with its local subgraph's representation obscure its original representation and retain its corresponding edges. For node v, we perform random walk on the bipartite graph G with its walk length setting as k. then we take the mean pooling of sampled nodes v's local subgraph's representation.
Edge-Dropping View The goal of the Edge- Bage-Droopping view is to generate subgraph filtering noisy edges and intentionally decreasing the influence of popular nodes for GCN layers. Similarly to the Node pooe-Dro view, create the Edge- elge-Dropping view by learnable edge dropping:
(1) (V, 3 Pij ei E E}},
where / (0, 13 also follows Pij Bern(u and denotes whether the edge eij is present.
5

---

Following (26), we adopt multi-layer perceptrons (MLPs) to the parameter w/ and wij that control the whether to mask node U; and edge eij, respectively, which can be formulated as:
= M ))); Wij MLP(1c (9 I)
(10)
To efficiently optimize the multi-view structure learning in an end-to-end manner, we adopt the reparameterizztion trick [II] and relax the above binary entries from being drawn from Bernoulli distribution to a deterministic function of parameter w and an independent random variable e, which can be formulate as:
= a(loge - log +c)++))++)
(11)
where e ~ Uniform(0) 1). E IR+ indicates the temperature and o()) is the sigmoid function. With T > O, the function is smoothed with a well-defined gradient 8u enabling efficient optimization of the learnable establishment of Node- Dropping view and Edge-Dropping view during training. In inference, we drop the node or edge with a probability of less than 0.5.
dp
Afterwards, we perform GCNs to obtain the representation of users and items on these views:
E (
NDD GCN(E (1-1) ND (NDD.. E GCN(E (1-1) ,
(12)
where the initial E NN = E (0)
ED E(,), After stacking L LightGCN layers, we also adopt the weighted sum to construct their final representation END and EED, respectively. For simplicity, we omit the augmentation type ND and ED in the symbols below, and use E to denote the representations of these augmentation views.
4.2 Information Bottleneck Contrastive Learning
Although we couple the learnable augmentation process and the recommendation process together, we find relying solely on the recommendation objective can not well guide the dropout process to create optimal augmentation views. Thus we adopt the Information-Botie retain the minimum sufficient information in each view for the downstream recommendation. Specifically, different from conventional contrastive learning, between the representations the augmentation view and the original graph while maximizing the information relevant to the recommendation task. By doing we can obtain comprehensive multi- view representation and efficiently drop noisy collaborative information for the recommendation. Accordingly, the objective in Eq. i induced
principle to
instead encourage the divergence
min Cree + I(E; E), (E,E)
(13)
where Erec is the BPR loss of the representation from the augmentation view and I(E, E) represents the mutual information between representations from two corresponding views.
According to 25 19 minimizing the InfoNCE loss [4] is equivalence to maximizing the lower bound of the corresponding mutual information. So adopt negative InfoNCE to estimate the mutual information between the representations of the augmentation view and the original graph, which consists of mutual information from both the user side and item side. Formally, for the user side mutual information, we consider the representations of the same users in the augmentation view and the original graph as the positive pairs (i.e., {(e; ē;) v; E U))), while representations of two different users in the augmentation view and the original graph as the negative pairs (i.e., {(e,, ē5) Ug By U + i});
erp(s(e,, ē,)/T')
T(E,; E..)
Σ log B,Eu
(14)
Σnjeu eap(s(c, 0,)/7')
where s(.) measures the similarity between two vectors, which is set as cosine similarity function; r' is the hyper-paramete parameter indicating the temperature similar to Eq. [1 Analogously, we can obtain the mutual information from item side [EE; E.) and the overall mutual information can be obtained by combining mutual information from two sides: (EE; E) (E.. E..) + [EE, E.).

---

4.3 Optimization
To obtain comprehensive multi-view representations, we utilize two parameterized networks to learn to create the Node- Dooe-Droppnng view and the Edge-Dropping view simultaneously. In order to integrally explore both views for better recommendation, we jointly optimize the recommendation tasks of these views and the self-superviser IB contrastive learning:
(15) where LND and CNB are the recommendation objective of the Node-Dropping view and Edge- Dropping view respectively. The last term is L2 regularization. À and 3 are the hyper-parameters controlling the effect strength of the IB contrastive learning task and L2 regularization, respectively.
L Crec + fre D. Cree + AI(E, + [E, EED)) + B||O||
Proposition 1. Formally, we denote the learned augmentation view as Ô, the noisy graph structure as G", and the downstream recommendation information as Y YRee. Suppose G' is irrelevant to YRe.s the mutual information g)) is upper bounded by ((; T(Ynec; 9):
( ((::)) T(YRoc:9))
(16)
Proof. Following the Markov chain assumption in 0. we suppose G is defined by Y and g'. And we can define the following Markov chain (Y Rees g') + G + G. According to the Data Processing Inequality, we have:
I(9:0) 2 I((Reee: g'): 9) ((Y Reci Glg')
(17)
= [()):)) + H - HY (Y
Since G' and Yice are independent, we have H(YRea (g") H(Ynec). Also, it's straightforward that H H(Y (Y 9) < M(YRoe)9) Thus we can simplify Eq. 17 as follow:
I(9: 9) > I(g': () HYRec)) H(Hrecl))) I(G': + I(YReci
(18)
Thus we obtain that (()': < I(G:)) I(Y 9). where [YYnec 9) is inverse proportional to the Erec in Eq. 13 Eq. proves that optimizing the IB contrastive objective in Eq. 13 is equivalent to minimizing the mutual information between the learned augmentation view and noisy structure. Specifically, it provides theoretical guarantees that the IB contrastive learning leads to the noise- invariance property by compressing the information in both the augmentation views. Mcanwhile, the IB contrastive objective also restricts the augmentation view to be predictive for the recommendation task, which can intentionally reduce the influence of popular nodes while preserving information of the isolated nodes, and thus help to mitigate the popularity bias.
5 Experiments
5.1 Experimental Setup
Three public available datasets are employed in our experiments, i.e.. Yelp2018, MovieLens-IM and Douban. The detailed description can be found in the Appendix. For each dataset, we randomly select 80% of the historical interactions of each user as the training set, 10% of those as the validation set, and the remaining 10% as the test set.
Dataset Description
Evaluation metrics To evaluate the performance of all methods, adopt ranking- based metric namely Normalized Discounted Cumulative Gain & (NDCG@ k) and a relevancy- based metric Hit Ratio@k A (RECALL k). The formulations of the metrics are in the Appendix. As suggested by Krichene and Rendle [3], we perform item ranking on all the candidate items instead of the sampled item sets to calculate above metrics, which guarantees that the evaluation process is unbiased.
Compared Methods We compare our CGI with three classes of baseline methods: (1) MF- based methods, i.e., BPRMF [77 and NCF 77]. (2) GNNs-based methods, i.e., NGCF (30) and LightGCN [8) and (3) CL-based methods, i.e., DNN+SSL (36) and SGL (31] We give detailed
introduction to these baselines in the Appendix. Note that DNN+SSL applies augmentation on items'
feature which is not applicable in case. So following B1), we apply the augmentations ID embeddings of items instead.
7

---

Table Comparison among models. Boldface denotes the highest score and underline indicates the best result of the baselines.
Yelp2018
Moviel.nns- IM
Model
NDCG@ 10
RECALL@10
NDC0@20
RECALL@20
NDCG@10
RECALLØ
NDCG@20
RECALL@20
BPRMP
0.0138
0.0208
0.0191
0.0373
0.1225
0.1376
0.1407
0.1882
NCF
0.0224
0.0356
0.0289
0.0566
0.1430
0.1546
0.1576
0.3027
NGCP
0.1462
0.1651
LighTGCN
0.0344
0.0530
0.0445
0.0850
0.1696
0.1865
0.1863
0,2420
DNN+SSI
0.0217
0.0344
0.0286
0.0564
0.1096
0.1238
0.1250
0.1714
SGL
0.0367
0,0552
0.0473
0.0891
0.1800
0.1965
0.1972
0,2520
CGI
0,0392
0,0584
0.0501
0,0932
0.1979
0,2180
0.2152
0.2772
Improw:
+6.82%
+5.90%
+5.93%
+4.58%
+9.95%
+10.91%
+9.13%
+9,97%
p-valae
1.29p-3
3,53e-3
7.00c-4
3.59c-
8.89c-4
4.220-4
4,83c-4
5.07e-5
Douban
Model
NDCG@ 10
RECALL@ 10
NDCG@20
RECALL@20
BPRMF
0.0496
0.0526
0.0516
0.0613
NCF
0.0694
0.0659
0.0734
NGCF
0.0794
0.0823
0.0784
0.0897
LightGCN
0.0862
0.0876
0,0845
0.0940
DNN+SSL
0,0712
0,0738
0,0703
0.0800
SGL
0.0912
0.0906
0.0910
0.1012
CGI
0.0991
0.1007
0.0979
0.1119
Improv.
+8.69%
+11.18%
+7.55%
+10.55%
p-value
1.99c33
4.40e-3
1.52o-4
1.60c-4
Hyper-parameter We initialize the latent vectors of both users and items with small random values for models. The parameters for baseline methods are initialized as the original papers, and are then carefully tuned to achieve optimal performances. For a fair comparison, the dimensions of both the user and item embeddings are all fixed to 64. We use Adam with 31 = 0.9, 82 = 0.999, 6 = 1e-8 to optimize all these methods. The batch size is set to 2048. The learning rate is set 0.005 and decayed at the rate of 0.9 every five epochs. We set À = 0.02 and 0.01 for the coefficients in More details about hyper-parameee settings of baselines can be found in the Appendix.
Eq. [5]
5.2 Performance Comparisons
We summarize the performance of different algorithms in terms of NDCG@k and RECALL@U (k- 10, 20) over three datasets in Table [ The experimental results demonstrate that CGI outperforms other methods all evaluation metrics. We conduct the significant test and p-values < 0.05 indicates that the improvements of our CGI are statistically significant.
Besides, we observe that the GNNs-based methods perform better than the MF-based models. These results verify that exploiting higher-order connectivity in the user-item bipartite graph is essential to improve the recommendation performance. This may also be the reason why the performance of DNN+SSL is inferior to those SGL and our CGI when all applying contrastive learning. We can see that the CL-based graph learning methods, including our CGI, consistently outperform the GNNs- based models, which verifies the effectiveness of contrastive learning for representation learning. Besides, CGI outperforms SGL by a large margin. The results demonstrate that compared with randomly dropping in SGL, the learnable graph augmentations optimized by information bottleneck can create optimal augmentation views and capture more comprehensive collaborative signals.
5.3 Ablation Studies
Effectiveness of Learnable Augmentation To understand the respective effects of both the node- dropping and edge-dropping in learnable augmentation, we conduct ablation studies on Yclp2018 and Moviclens- M.. As shown in Table 2 we report NDCG@ @ 10 and RECALL @ 10 of CGI and SGL in different versions. Specifically, CGIND and CGLED denote CGI with only node-dropping view and edge-dropping view being adopted, respectively. SGL D and SGL denotes the augmentation view in SGL is created by random node dropout and edge dropout, respectively.
We find that: (1) Our CGI achieves obvious improvements compared with SGL in different types of augmentation, which again verifies the effectiveness of the learnable graph augmentation optimized by information bottleneck. (2) CGI performs better in both CGI-ND and CGI-ED. We ascribe these to the ability of multi -iet learning, which enables the final representation to capture collaborative information of different semantics and thus enhances the robustness and expressiveness of the model.
8

---

Yelp2018
Yelp2018
Movielens-11 11
Model
NDCG@10
RECALL 10
model
del
LightGCN
0.0344
0,0530
0.05
LIghTCN
SGL
SGL
LightECN
CGI
0.0392
0.0584
CGI
CO
SGL-ND
0.0356
0.0544
0.01
CGI-ND
0.0369
0.0569
SGL.ED
0.0367
0.0552
CGI-ED
0.0379
0.0579
Rocal
0.03
MovieLens-IM
Model
NDCG@
RECALL@
0.02
LightGCN
0.1696
0.1865
CGI
0.1979
0.2180
SGL-ND
0.1765
0.1948
0.1
CGI-ND
0.1934
0,2119
SGL-ED CGI-ED
0,1800 0,1916
0.1965
0,2088
Item Group
Item Group
Table 2: Comparison among models.
Figure 3: Performance of different item groups
Par:
6441
Nohe Rati
Noke Sadi
259 213
Figure 4: Performance comparison over different noise ratio. The bar represents the NDCG@ 10 and the line on Yelp2018 represent the performance degradation ratio.
Figure 5: Effect of Information Bottleneck
(3) The performance of CGI-ED is better than that of CGI-ND in the sparse dataset Yelp2018, while worse in the dense dataset Movielens-IM. We can speculate that the interaction noises more significant in the sparse dataset with less useful information, in which CGI-ND is not so flexible. Because it will remove all influence (i.e., edges) popular nodes, which is hard be restored with scarce interactions. But in the dense dataset, popularity bias becomes more significant, which makes CGI-ND more efficient by blocking the influence from popular users or items.
Accuracy against Popularity Bias To verify whether CGI is capable of mitigating popularity bias, We split the item set I into groups (1-5) evenly based on their popularity. The larger the GroupID is, the larger degrees the items have. Following [31], we decompose the RECALL @ 10 metric of the whole dataset into the contributions the above ten groups of items:
RECALL (0) Σ rel.
(19)
[T
where rel, denotes the item at the rank i is in the test set and g-th item group at the same time. As such, RECALL(3) measures the performance over the g-th item group. From Fig. 3 we can see that recommender systems tend to recommend popular items, while leaving unpopular items less likely to be discovered, which further exacerbates the long-tail distribution. Also. our CGI can significantly improve the recommendation accuracy on long-tail items. Although both GCL methods CGI and SGL,, show no superiority on the top 20% items, from the overall improvements in Table [] we can see they can better capture the long-tail items' information user preference representations.
Robustness to Interaction Noises To verify CGI's robustness to interaction noises, we generate dif- ferent proportions of negative interactions (i.e., 5%, 10%, 15%, and 20%) to contaminate the training set, and report the performance on the unchanged test set. Fig. 4 shows the NDCG@ 10 on Yelp2018 and Movielens- M and the performance degradation ratio of the corresponding contaminated training set. It's obvious that the more noise we add, the worse performance all the models yield, since all the models utilize LightGCN as the basic backbone, which fully relies on the adjacency matrix Ag to refine the representations of users and items in Eq. However, the performance degradation of

---

Table Performance with Other GNN variants.
Yelp2018
Movielens. 1M
Douhan
Model
NDCG@ 10
Recull@ 10
NDCG@ 10
Recall@ IO
NDCG@ 10
Recall 10
GC-MC
0.0214
0.0278
0,1350
0.1491
0.0671
0.0739
SGL+GC-MC
0.0218(+1.99)
0.0281(+1.2%%)
0.1412(+4.66%
0.1577(+5.8%)
0.0687(+2.35)
0.0762(+3.1%)
CGI-GC-MC
0.0218(42.1%)
0.0282(+1.7%)
0. 0.1422(+5.3%)
0.1585(+6.3%)
0.0687(+2.3%)
0.0765(+3.55%)
NGCF
0.0242
0.0384
0,1462
0.1631
0.0794
0.0823
SGLANGGFF
0.0461(+774%8)
0.0418(+8.99)
0.1609(+10.155
(.1871(+13.39)
0.0833(+4.999)
0.0857(+4.11 (%)
CGI+NGCF
0.0272(+12.55%
0.0431(12.1%)
0.1660((11..6%)
0.1937(-17.3%)
0,0840(+5.75)
0.0875(+6.3%))
our CGI is smaller than other models in both datasets. What's more, the gaps between CGI and other models grow larger as the noise increase. This suggests that our CGI framework can mitigate the noise in interaction data more efficiently, and our learnable augmentation optimized by the IB contrastive learning exhibits good robustness in the presence of a high proportion of noise. which is consistent with our proof in Sect. 4.3 We can observe that CGI is more robust on Movielens- IM. This makes sense since Movielens-IMM 1M is much denser than Yelp2018 8 according to the statistics in the Appendix and thus the bipartite graph of Yelp2018 will be more sensitive the added noise.
Effectiveness of Information Bottleneck To investigate the effect of information bottleneck, consider the following variants of CGI with different contrastive learning strategies, our complete methods (CGI), our method without introducing contrastive learning (GL), and our method that maximizes the correspondence among different views (i.e., min Lrec [E; E)) (GCL). Fig. shows the recommending training loss w.z.. the number of training steps and the evaluation results on Yelp. from which we observe that the multi-view graph learning frameworks driven by contrastive learning are easier to converge. Specifically, when maximizing the mutual information among views, the GCL framework drops more quickly at the very beginning and turns to a steadily decreasing state afterward. However, with IB contrastive learning, the recommending loss of our CGI appears to have a declining trend after an initial sharp drop, instead of getting an early-stop, which more likely to converge to better local optimum. This is probably why CGI has better performance than both GL and GCL, as illustrated by the right part of Fig. 5 Also, we find that the multi-view graph learning can benefit more from the IB contrastive learning than the conventional one, since it can encourage to drop the noisy information irreverent for the recommendation as illustrated in Sect. 4.3
To verify the generalization of our method on other GNNs, we other popular LANN-based recommenders GC-MC 24 and NGCF 30 The experimental results are shown in Table 3, Both graph contrastive learning methods have shown improvements the backbones. On NGCF, CGI shows consistent superiority compared to SGL. On GC-MC, CGI does not have significant improvement compared to SGL., probably due to the fact that GC-MC only utilizes one layer of GCN, which means can only adopt 1-hop neighbors for learning, thus making the learnable augmentation challenging to fetch enough information.
Performance with Other GNNs
tried CGT and the baseline SGL
Conclusions
In this paper, we propose novel Contrastive Graph Structure Learning via Information Bottleneck (CGD) to learn better augmentation from different aspects for the multi-view representation learning of recommendation. In particular, propose fully differentiable learner to drop nodes and edges to construct different types of augmentation views coupled with the recommendation. We innovatively integrate information bottleneck into the multi-view contrastive learning process for recommendation and prove its efficiency. The extensive experiments conducted on three public datasets verify the effectiveness of CGI.
Acknowledgments and Disclosure of Funding
This work was supported by Alibaba Group through Alibaba Research Intern Program.
10

---

References
[1] A. Achille and S. Soatto. Emergence of invariance and disentanglement in deep representations.
J. Mach. Learn, Res., 19:50:1-5::3, 2018.
[2] M. Federici, A. Dutta, P. Forré, N. Kushman, and Z. Akata. Learning robust representations via multi-view information bottleneck. In 8th International Conference on Learning Representa- tions, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[3] M. Gori and A. Pucci. Itemrank: A random-wall based scoring algorithm for recommender engines. In IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, January 6-12, 2007, pages 2766-2771, 2007.
[4] M. Gutmann and A. Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010, volume 9, pages 297-304, 2010.
[5] W. L. Hamilton, Z. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024-1034, 2017.
[6] K. Hassani and A. H. K. Ahmadi. Contrastive multi-view representation learning on graphs. In Proceedings of the 37th International Conference on Machine Learning. ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 4116 4126, 2020.
[7] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth. Australia, April 3-7, 2017, pages 173-182, 2017.
[8] X. He. K. Deng, X. Wang, Y. Li. Y. Zhang, and M. Wang. Lightgen: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pages 639-648, 2020.
[9] R. D. Hjelm, A. Fedorov,
Maaoiie-arrchildo. K. Grewal, Bachman, A. Trischler, and Y. Bengio. Learning deep representations by mutual information estimation and maximization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[10] W. Hu, Liu, J. Gomes, M. Zitnik, P. Liang, V. Pande, and Leskovee. Strategies for pre- training graph neural networks. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[11] E. Jang, S. Gu, and B. Poole. Categorical reparameterization with gumbel-softmax. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26. 2017, Conference Track Proceedings, 2017.
[12] N. Kipf and M. Welling. Supi-supe classification with graph convolutional networks. In 5th International Conference Leurning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Truck Proceedings, 2017.
[13] W. Krichene and S. Rendle. On sampled metrics for item recommendation. In KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 1748-1757, 2020.
[14] Lian, X. Zhou, F. Zhang, Chen, Xie, and Sun. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, KDD 2018, London, UK, August 19-23, 2018, pages 1754-1763, 2018.

---

[15] Z. Peng, W. Huang, M. Luo, Q. Zheng, Y. Rong, T. Xu, and J. Huang, Graph representation learning via graphical mutual information maximization, In WWW '20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pages 259-270, 2020.
[16] J. Qiu, Q. Chen, Y. Dong, J. Zhang, H. Yang, M. Ding, K. Wang, and Tang. GCC: graph contrastive coding for graph neural network pre-training. In KDD 20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pages 1150-1160, 2020.
[17] S. Rendle, C. Freudenthaler, Z. Gantner, and L. Tehmmed-Thhime. BPR: bayesian personalized ranking from implicit feedback. In UAI 2009, Proceedings of the Twenty-Fifih Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada, June 18-21, 2009, pages 452-461, 2009.
[18] Y. Saito. S. Yaginuma, Y. Nishino, H. Sakata, and K. Nakata. Unbiased recommender learning from miising--ot--t--andde implicit feedback. WSDM '20: The Thirteenth ACM Interna- lional Conference on Web Search and Data Mining, Houston, TX, USA, February 3-7, 2020, pages 501-509, 2020.
[19] F. Sun, J. Hoffmann, V. Verma, and J. Tang. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020, 2020.
[20] Q. Sun, Li, H. Peng, J. Wu, X. Fu, Ji,, and P. S. Yu. Graph structure learning with variational information bottleneck. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourts Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22- March 1, 2022, pages 4165-4174, 2022.
[21] S. Suresh, P. Li, C. Hao, and J. Neville. Adversarial graph augmentation to improve graph contrastive learning. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, 2021.
[22] Y. Tian, D. Krishnan, and P. Isola. Contrastive multiview coding. In Computer Vision ECCV 2020 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XI, volume 12356, pages 776-794, 2020.
[23] N. Tishby, F. C. N. Percira, and W. Bialek. The information bottleneck method. CoRR, physics/0004057, 2000.
[24] R. den Berg, T. N. Kipf, and M. Welling. Graph convolutional matrix completion. CORR, abs/1706.00263. 2017.
[25] A. van den Oord, Li, and O. Vinyals. Representation learning with contrastive predictive coding. CoRR, abs/1 887 .0774,, 2018.
[26] P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3, 2018, Conference Track Proceedings, 2018
[27] Velickovic, W. Fedus, W. L. Hamilton, Liò, Y. Bengio, and R. D. Hjelm. Deep graph infomax. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[28] Z. Wan, C. Zhang, P. Zhu, and Q Hu. Multi-view In Thirt--Fifh AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Thire Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, FAAI 2021, Virtual Event, February 2-9, 2021, pages 10085-10092, 2021.
representation learning.
12

---

[29] Q. Wang. C. Boudreau, Q. Luo, P. Tan, and J. Zhou. Deep multi-view information bottleneck. In Proceedings of the 2019 SIAM International Conference on Data Mining, SDM 2019, Calgary, Alberta, Canada, May 2-4, 2019, pages 37-45, 2019.
[30] X. Wang. X. He, M. Wang, F. Feng, and T. Chua. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, July 21-25, 2019, pages 165-174, 2019.
[31] J. Wu, X. Wang, F. Feng, X. He, L. Chen, J. Lian, and X. Xie. Self-supervised graph learning for recommendation. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, pages 726-735, 2021.
[32] T. Wu, H. Ren, P. Li, and J. Leskovec. Graph information bottleneck. In Advances in Neural Information Processing Systems 33: Anmual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12. 2020, virtual, 2020.
[33] X. Xia, H. Yin, J. Yu, Y. Shao, and L. Cui, Self-supervieed graph co-training for session-based recommendation. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 5. 2021, pages 2021.
[34] C. Xu, D. Tao, and C. Xu. Large-margin multi-viewinformation bottleneck. IEEE Trans. Pattern Anal. Mach. Intell.,
2014.
[35] K. Xu, W. Hu, J. Leskovec, and S. Jegelka. How powerful are graph neural networks? In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019, 2019.
[36] T. Yao, X. Yi, D. Z. Cheng, F. X. Yu, A. K. Menon, L. Hong, E. H. Chi, S. Tjoa, J. Kang, and E. Ettinger. Sulf-super learning for deep models recommendations. CoRR, abs/2007. 12865, 2020.
[37] R. Ying, R. He, K. Chen, P. Eksombatchai, W. L. Hamilton, and J. Leskovec. Graph con- volutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, pages 974-983, 2018
[38] Y. You, T. Chen, Y. Sui, T. Chen, Z. Wang, and Y. Shen. Graph contrastive learning with augmentations. In Advances Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 12, 2020, virtual, 2020.
[39] Y. You, Т. Chen, Z. Wang, and Y. Shen. Bringing your own view: Graph contrastive learning without prefabricated data augmentations. In WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining, Virtual Event / Tempe, AZ, USA, February 21 25, 2022, pages 1300-1309, 2022.
[40] Yu, T. Xu, Y. Rong, Y. Bian, Huang, and R. He. Graph information bottleneck for subgraph recognition. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021, 2021.
[41] J. Yu, H. Yin, Li, Q. Wang. N. Q V. Hung. and X. Zhang. Self-supervised multi-channel hy- pergraph convolutional network for social recommendation. In WWW '21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 413-424, 2021.
[42] J. Yu, Cao, and R. He. Improving subgraph recognition with variational graph information bottleneck. pages 19405, 2022.
[43] G. Zhou, X. Zhu, C. Song, Fan, H. Zhu, X. Ma, Yan, J. Jin, H. Li, and K. Gai. Deep interest network for tlick-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK August 19-23, 2018, pages 1059-1068, 2018.
13

---

[44] D. Zhu, Z. Zhang, P. Cui, and W. Zhu. Robust graph convolutional networks against adversarial attacks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pages 1399- 1407, 2019.
[45] Z. Zhu, Y. He, X. Zhao, and J. Caverlee. Popularity bias in dynamic recommendation. In KDD 21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Virtual Event, Singapore, August 14-18, 2021, pages 2439-2449, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] We discuss them in Appendix
(c) Did you discuss any potential negative societal impacts of your work? [Yes] We discuss them in Appendix
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes] Our codes are made publicly available github.
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes] We specify all training details in Section 5.1 and Appendix
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] See Appendix. We run experiments multiple times and report statistical significance in Section 5 consistent with previous works.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] We discuss them in Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [No] The code and the data are public.
(c) Did you include any new assets either in the supplemental material or as URL? [Yes] Our codes shared using URI.
(d) Did you discuss whether and how consent was obtained from people whose data you'le rou're using/curating? [No] The code and the data are public.
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "oeV4gKJI3hv", "reviewer": "Reviewer_wqw8", "summary": "This paper presents a new deep learning architecture for graphs, named CGI. CGI aims at build a vector space for users and items that can be used for recommander systems. The main novelty lies in the way two views are built (one for the nodes, one for the edges), by using data augmentation with perturbation (for dealing with high-degree nodes) and Information Bottleneck framework to make the representation space more robust to noise. CGI is favorably compared to several recent models on three different datasets with a proper experimental framework (good metrics, ablation studies, etc.). I don't see any import flaw in the methodology. All in all, it seems to be as a good, rational contribution to the field.", "questions": "Is it possible to design a more accurate baseline by mitigating the weight of node centrality (like a normalization) before using solutions from the literature?\n\nSection 4.3 is not perfectly clear to me. I don't understand to what extent Proposition 1 is a real new contribution (that can be used for other research, and in what context), or if it's a rather straightforward derivation.", "limitations": "I don't see any important limitation. I've not given the maximal score because it's really interesting but not fundamentally novel (like, \"groundbreaking\").\n\nHowever, I'm bothered by the fact the authors don't point out any possible limitation... and I cannot believe that. From my opinion, a paper like this should tell when the model isn't fit to the problem and what may be the possible extensions. It's maybe a problem of the (possible) computation cost of optimization, or when the two views (node vs. edge) disagree?", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- new interesting model with good motivations\n- good results when compared to recent sota\n \nWeaknesses:\nI don't see any important weakness.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "dLWfvebzgOH", "reviewer": "Reviewer_UmNd", "summary": "This paper proposes CGI, a contrastive model for item-user bipartite graphs in recommendation systems. The CGI consists of two parts: learnable graph structure augmentation and information bottleneck losses. Each of these modules reduces the influence of popularity biases and noisy interactions. The experiments on three real-world recommendation tasks demonstrate that CGI outperforms baselines, especially for unpopular items and noisy settings.\n", "questions": "- It is nice to see the statistical significance tests in the result tables. Could you write the standard deviation and the number of actual runs in the experiments?\n- How about changing the name from CGI to CGIB? Surveying existing works, models that end with ‘I’ are conventionally InfoMax models (DGI, HDGI, GMI, GCI, …) and models that end with ‘IB’ are usually Information bottleneck models (GIB, HGIB, VIB-GSL, VGIB, …).\n", "limitations": "The authors fairly discussed limitations.\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Strengths\n\nThis paper deals with an important problem in recommendation systems. The model design aligns with the motivation, and extensive studies under various scenarios are well-presented. The authors design experiments for sub-problems of this particular domain (popularity biases and interaction noises) and present them in an appropriate format. I like the Effectiveness of Information Bottleneck part (line 340) since this is an essential analysis to support the model design choice of MI minimization rather than maximization.\n\n## Weaknesses\n\nAlthough I have decided to rate this paper as borderline accept, there are two flaws in this paper. One is a minor problem that can be solved by simple revision, but the other can be critical. Even if a rejection decision is made, I will not object to it.\n\nFirst, some related works of information bottleneck models on graphs are missing.\n\n- Yu, Junchi, et al. \"Graph Information Bottleneck for Subgraph Recognition.\" International Conference on Learning Representations. 2020.\n- You, Yuning, et al. \"Bringing your own view: Graph contrastive learning without prefabricated data augmentations.\" Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. 2022.\n- Sun, Q., et al. \"Graph Structure Learning with Variational Information Bottleneck.\" Proceedings of the AAAI Conference on Artificial Intelligence. 2022.\n- Yu, Junchi, Jie Cao, and Ran He. \"Improving subgraph recognition with variational graph information bottleneck.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. (Yes, I know this paper is very recently published, but it was on arXiv more than three months ago. Moreover, it is deeply related to the authors' work. See the 'Second' section below)\n\nSecond, the CGI model might be a straightforward combination of existing approaches but for a specific domain. The technical novelty is somewhat limited. Learnable masking of nodes or edges with reparameterization is a well-known approach for learning graph structures, and using it as augmentations have been proposed recently (as the authors say). The authors use information bottleneck losses in the same form as the original GIB paper (and other variants for graph tasks) with similar motivations. I believe this is an appropriate choice for this problem, but it is difficult to call it novel. In addition, the theoretical result (proposition 1) is a simple application of Proposition 3.1 in [1] and is very similar to Lemma 4.1 in [2].\n\n[1] Achille, Alessandro, and Stefano Soatto. \"Emergence of invariance and disentanglement in deep representations.\" The Journal of Machine Learning Research 19.1 (2018): 1947-1980.\n\n[2] Yu, Junchi, Jie Cao, and Ran He. \"Improving subgraph recognition with variational graph information bottleneck.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "Of7CAITmzI5", "reviewer": "Reviewer_cHS4", "summary": "This work aims to alleviate the effect of popularity bias and noisy user-item interactions in graph convolutional networks (GCNs) for recommendation. To this end the authors propose a GCN-based recommender method called Contrastive Graph Structure Learning via Information Bottleneck (CGI). In addition to the standard GCN layer producing node representations from the observed user-item graph, CGI includes a node and an edge dropping components whose role is to generate augmentations of the observed graph by dropping some nodes and edges respectively. To improve the quality of the learned augmentations, the authors introduce (in addition to the recommendation objective) an Information Bottleneck (IB)-based contrastive objective, which encourages the representations learned from the augmented graphs to be independent from those learned from the observed graph. The proposed method is evaluated and compared to some existing recommender models on three real-world datasets.  ", "questions": "Please refer to the strengths and weaknesses section. ", "limitations": "Yes.", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths.\n1. The paper is well written and technically sound. \n2. This work investigates interesting ideas such as the IB-based contrastive objective, as well as learning the graph augmentations instead of relying on random or predefined procedures to generate them. \n3. Empirical results show that CGI outperforms the chosen baselines on the recommendation task and seems more robust to noisy user-item interactions.\n\nWeaknesses.\n1. The claim regarding the ability of the proposed method to alleviate the popularity bias is not well supported in the paper, nor by theoretical analyses, neither by convincing targeted experiments. For examples, I would recommend reporting statistics about the popularity distribution of the recommended items by the different baselines. Also, some quantitative and qualitative experiments on how popular/rare items are ranked by the different models for a given set of users would be meaningful (see for instance Figures 2 and 3 in [1]).\n2. Experiments are weak. Three datasets are considered for evaluation: Yelp, MovieLens and Douban. However, most of the results are on Yelp and/or MovieLens, except in Table 1.  I would recommend reporting the results of every experiment across all the three datasets.\n\nAdditional comments/questions.\n\n- In section 4.3, the part related to proposition 1 is a bit hard to follow and connect to the objective of eq. 13. This is due to using different notations for the mutual information terms in the proposition and in eq. 13. Please consider improving the notations.  \n- I would recommend keeping the legend consistent across all the experiments. For instance, CGI is represented by a green bar in figures 3 and 4, while in figure 5 it is represented by a blue bar.  \n- What type of significance test is used in the experiments, and how many trials are performed for every algorithm in Table 1. \n\nReferences.\n\n[1] Liang, Dawen, et al. \"Factorization meets the item embedding: Regularizing matrix factorization with item co-occurrence.\" Proceedings of the 10th ACM conference on recommender systems. 2016.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "AFDBEBX8bXz", "reviewer": "Reviewer_5uX4", "summary": "The paper proposes a graph neural network model based on LightGCN using learned parameters to adaptively dropout nodes and edges to obtain multi-view representation and contrastive learning with information bottleneck. Experiments on public datasets show improvements compared to existing models. Ablation studies show effectiveness of different components in the model.", "questions": "The main GNN component is based on LightGCN, have you tried with other GNN variants? Do they work well?\n\nFrom the details of the dataset, it seems that the paper only considers a binary case for Movielens and Douban (1 vs 0) and this seems to simplify the problem. For real recommendation systems, there's a different between recommending an item that user would rate 5 vs 4 (assuming 5 is the highest rating). It would be worth trying to fully utilize the ratings information in these datasets.\n\nIf the proposed model aims to address the popularity bias issue, why are items with fewer interactions removed (Yelp with <10, Movielens with <3, Douban with <5)? These are real data and should be considered in the model training and evaluation unless they are corrupt or spammy data points.", "limitations": "The model assumes that the data is relatively clean (without much spam or misleading content/item). Real recommendation systems would need to consider filtering out spammy content, misleading and low quality content. ", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n- The paper explains the motivation behind different model components, e.g., adaptive dropping of edges and nodes to reduce popularity bias and contrastive learning via information bottleneck to reduce noise.\n- The paper conducts thorough experiments using 3 public datasets and compared with multiple models. There are also ablation studies to verify different hypothesis in the model. The paper also provides detailed hyperparameters and time complexity analysis. Code was also provided for reproducing the results.\n\nWeakness\n- While the paper provides empirical results showing that the model can reduce popularity bias and provide robustness to noise, there are no clear theoretical link provided.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept