Inductive Collaborative Filtering via Relation Graph Learning

1 INTRODUCTION . As information explosion has become one major factor affecting human life in the decade , recommender systems , which can filter useful information and contents of user ’ s potential interests , play an increasingly indispensable part in day-to-day activities . Recommendation problems can be generally formalized as matrix completion ( MC ) where one has a user-item rating matrix whose entries , which stand for interactions of users with items ( ratings or click behaviors ) , are partially observed . The goal of MC is to predict missing entries ( unobserved or future potential interactions ) in the matrix based on the observed ones . Modern recommender systems need to meet two important requirements in order for desirable effectiveness and practical utility . First of all , recommendation models should have enough expressiveness to capture diverse user interests and preferences so that the systems can accomplish personalized recommendation . Existing methods based on collaborative filtering ( CF ) or , interchangeably , matrix factorization ( MF ) have shown great power in this problem by factorizing the rating matrix into two classes of latent factors ( i.e. , embeddings ) for users and items respectively , and further leverage dot-product of two factors to predict potential ratings ( Koren et al. , 2009 ; Rendle et al. , 2009 ; Srebro et al. , 2004 ; Zheng et al. , 2016b ) . Equivalently , for each user , the methods consider a one-hot user index as input , assume a user-specific embedding function ( which maps a user index to a latent factor ) , and use the learnable latent factor to represent user ’ s preferences in a low-dimensional space . One can select proper dimension size to control balance between capacity and generalization . Recent works extend MF with complex architectures , like multi-layer perceptrons ( Dziugaite & Roy , 2015 ) , recurrent units ( Monti et al. , 2017 ) , autoregressive models ( Zheng et al. , 2016a ) , graph neural networks ( van den Berg et al. , 2017 ) , etc. , and achieve state-of-the-art results on most benchmarks . The second requirement stems from a key observation from real-world scenarios : recommender systems often interact with a dynamic open world where new users , who are not exposed to models during training , may appear in test stage . This requires that models trained on one group of users manage to adapt to unseen users . However , the above-mentioned CF models would fail in this situation since the user-specific embeddings are parametrized for specific users and need to be learned collaboratively with all other users in transductive setting . One brute-force way is to retrain the whole model with an augmented rating matrix , but extra time cost would be unacceptable for online systems . There are quite a few studies that propose inductive matrix completion models using user features ( Jain & Dhillon , 2013 ; Xu et al. , 2013 ; Cheng et al. , 2016 ; Ying et al. , 2018 ; Zhong et al. , 2018 ) . Their different thinking paradigm is to target a user-sharing mapping from user features to user representations , instead of from one-hot user indices used by CF models . Since the feature space is shared among users , such methods are able to adapt a model trained on existing users to unseen users . Nevertheless , feature-driven models often suffer from limited expressiveness with low-quality features that have weak correlation with target labels . For example , users with the same age and occupation ( commonly used features ) may have distinct ratings on movies and music . Unfortunately , high-quality features that can unveil user interests for personalized recommendation are often hard to collect due to increasingly concerned privacy issues . A following question arises : Can we have a recommendation model that guarantees enough expressiveness for personalized recommendation and enables inductive learning ? In fact , to simultaneously meet the two requirements is a non-trivial challenge when high-quality user features are unavailable . First , to achieve either of them , one needs to compromise on the other . In fact , the one-hot user indices ( together with learnable user-specific embeddings ) give a maximized capacity for learning distinct user preferences from historical rating patterns . To make inductive learning possible , one needs to construct a shared input feature space among users out of the rating matrix , as an alternative to one-hot user indices . However , the new constructed features have relatively insufficient expressive power . Second , the computation based on new feature space often bring extra costs for time and space , which limits model ’ s scalability to large-scale datasets . In this paper , we propose an inductive collaborative filtering model ( IRCF ) 1 as a general CF framework that achieves inductive learning for matrix completion and meanwhile guarantees enough expressiveness and scalability . As shown in Fig . 1 , we consider a base transductive matrix factorization model trained on one group of users ( called support users ) and a relation inference model that aims to estimate their relations to another group of users ( called query users ) w.r.t . historical rating patterns . The ( multiple ) estimated relational graphs enable attentively message passing from users to users in the latent space and compute user-specific representations in an inductive way . The output user representations can be used to compute product with item representations to predict ratings in the matrix , as is done by CF models . Compared with other methods , one key advantage of IRCF is the capability for inductively computing user-specific representations without using features . Besides , our method possesses the following merits . 1 ) Expressiveness : A general version of our model can minimize reconstruction loss to the same level as matrix factorization under a mild condition . Also , we qualitatively show its superior expressiveness than feature-driven and local-graphbased inductive models that may fail in some typical cases . Empirically , IRCF provides very close performance to transductive CF models when given sufficient training ratings . 2 ) Generalization : IRCF manages to achieve state-of-the-art results on new ( unseen ) users compared with inductive models . Also , IRCF gives much better accuracy than transductive models when training data becomes sparse and outperforms other competitors in extreme cold-start recommendation . 3 ) Scal- 1The codes will be released . ability : Our model enables mini-batch training and efficient inference . In experiments , IRCF is averagely ten times faster than local-graph-based inductive model . 4 ) Flexibility : As a general CF framework , IRCF is flexible to incorporate with various architectures ( e.g . MLP-based , GNN-based , autoregressive , etc . ) as a base transductive model as well as deal with implicit user feedbacks . 2 BACKGROUND AND RELATED WORKS . In this section , we present some background of the problem and discuss relationships to related works . We consider a general matrix completion ( MC ) problem which deals with a user-item rating matrix R = { rui } M×N where M and N are the numbers of users and items , respectively . For implicit feedback , rui is a binary entry which denotes whether user u rated ( or clicked on , reviewed , liked , purchased , etc . ) item i or not . For explicit feedback , rui records rating value of user u on item i . The entries of R are partially observed and the goal is to estimate the missing values in the matrix2 . In the following , we introduce related works and highlight their differences to our paper . In Fig . 5 , we provide an illustration for comparison with these methods . General Collaborative Filtering . Existing methods for MC are generally based on collaborative filtering ( CF ) or , interchangeably , matrix factorization ( MF ) where user u ( resp . item i ) corresponds to a d-dimensional latent factor ( i.e. , one-hot embedding ) pu ( resp . qi ) .Then one has a prediction model r̂ui = f ( pu , qi ) where f can be basically specified as simple dot product or some complex architectures , like neural networks , graph neural networks , etc . One advantage of CF models is that the user-specific embedding pu ( as learnable parameters ) can provide enough expressive power for learning diverse personal preferences from user historical behaviors and decent generalization ability through collaborative learning with all the users and items . Furthermore , the user embeddings possess rich profile information as representation of user preferences and can benefit various downstream tasks , like target advertisement , user-controllable recommendation ( Ma et al. , 2019 ; Cen et al. , 2020 ) , influence maximization ( Khalil et al. , 2017 ; Manchanda et al. , 2019 ) , friend recommendation , etc . However , such user-specific embedding limits the model in transductive learning and when it comes to new users during test , one has to retrain the model ( often associated with embeddings for both new and existing users as well as items ) with new augmented rating matrix . Admittedly , one can consider local updates for embeddings of new users with fixed item embeddings learned from existing users . Nevertheless , such operation makes model learning for users independent from each other , which is prone for over-fitting compared with collaborative learning in CF models . Besides , it requires ‘ incremental ’ learning for each new user , while IRCF can deliver on-the-fly inference in online systems . Feature-driven Recommendation . The CF models do not require any side information other than the rating matrix , but can not be trained inductively due to the learnable user-specific embedding pu . To address the issue , one can leverage side information such as attribute features to achieve inductive learning . Define user features ( like age , occupation , etc . ) as au and item features ( like movie genre , director , etc . ) as bi . The feature-driven model targets a prediction model r̂ui = g ( au , bi ) . Since the space of au is shared among users , a model trained on one group of users can adapt to other users without retraining . However , feature-driven models often provide limited performance since the shared feature space is not expressive enough compared to one-hot embedding space . Another issue is that high-quality features are hard to collect in practice . We note that our model does not require user features for inductive collaborative filtering . Inductive Matrix Completion . There are a few existing works that attempt to handle inductive matrix completion using only user-item rating matrix . ( Hartford et al. , 2018 ) ( F-EAE ) puts forward an exchangeable matrix layer that takes a whole rating matrix as input and inductively outputs prediction for missing ratings . However , the scalability of F-EAE is limited since it requires the whole rating matrix as input for training and inference for users , while IRCF enables mini-batch training and efficient inference . Besides , ( Zhang & Chen , 2020 ) ( IGMC ) proposes to use local subgraphs of user-item pairs in a bipartite graph of rating information as input features and further adopt graph neural networks as representation tool to encode subgraph structures for rating prediction . The model achieves inductive learning via replacing users ’ one-hot index embeddings by shared input features ( i.e. , index-free local subgraph structures ) . Differently , IRCF maintains the ability to give 2Also , in some situations , one targets a ranking list of items for each user as top-N recommendation . In this paper , we focus on predicting missing values in the matrix and leave top-N recommendation for future works . user-specific embeddings , which represent users ’ preferences and interests and can be used for downstream tasks ( like target advertisement , user-controllable recommendation , influence maximization , etc . ) while F-EAE and IGMC can merely output a prediction score without such user representation . Moreover , the expressiveness of IGMC is limited since the local subgraph structures can be indistinguishable for users with distinct behaviors ( see Section 3.2 for more discussions ) , while IRCF has equivalent expressiveness as original CF models . Item-based CF Models . Some previous works use item embeddings as representation for users . ( Cremonesi et al. , 2010 ; Kabbur et al. , 2013 ) proposes to use a combination of items rated by users to compute user embeddings and frees the model from learning parametric one-hot embeddings for users . Furthermore , there are quite a few auto-encoder architectures for recommendation problem , leveraging user ’ s rating vector ( ratings on all the items ) as observed input , estimate user embedding ( as latent variables ) based on that , and output prediction for missing values in the rating vector ( Sedhain et al. , 2015 ; Liang et al. , 2018 ) . With item embeddings and user ’ s rating history , these methods achieve inductive learning for users and can adapt to new users during test . On methodological level , IRCF has the following differences : 1 ) IRCF learns to use weighted combination of users ’ embeddings to compute embeddings for new users , and such combination weights possess interpretability for underlying social influence or user proximity ; 2 ) IRCF considers both users ’ and items ’ one-hot embeddings in general CF models that maintain better capacity than item-based CF models that only considers learnable parameters in item embedding space .

Summary: This work proposed an inductive recommendation framework on user-item relation graphs. Such a framework relies on the user-item relations without the requirement of side-information and perceives certain flexibility in terms of the parametrization for user/item representations. The authors also provided theoretical analysis to highlight some mathematical insights out of this framework. The proposed method is evaluated on three real-world datasets and compared with several baselines.