Incorporating User-Item Similarity in Hybrid Neighborhood-based Recommendation System

Modern hybrid recommendation systems require a sufficient amount of data . However , several internet privacy issues make users skeptical about sharing their personal information with online service providers . This work introduces various novel methods utilizing the baseline estimate to learn user interests from their interactions . Subsequently , extracted user feature vectors are implemented to estimate the user-item correlations , providing an additional fine-tuning factor for neighborhood-based collaborative filtering systems . Comprehensive experiments show that utilizing the user-item similarity can boost the accuracy of hybrid neighborhood-based systems by at least 2.11 % while minimizing the need for tracking users ’ digital footprints . 1 INTRODUCTION . The continuously accelerated growth of communication technology and data storage in the past decades has benefited customers with an enormous amount of online multimedia content such as movies , music , news , and articles , creating billion-dollar industries . Following this evolution , recommendation systems ( RSs ) have been widely developed to automatically help users filter redundant information and suggest only suitable products that fit their needs . Such systems are used in a variety of domains and have become a part of our daily online experience ( Ricci et al. , 2015 ) . RSs are commonly classified into three main types ( Adomavicius & Tuzhilin , 2005 ) : the contentbased technique , the collaborative filtering technique , and the hybrid technique . The content-based approach , as in Lang ( 1995 ) ; Pazzani & Billsus ( 1997 ) ; Lops et al . ( 2011 ) ; Narducci et al . ( 2016 ) , learns to recommend items that are similar to the ones that a user liked based on items ’ features . The main weakness of this approach is the lack of available and reliable metadata associated with item ( Mooney & Roy , 2000 ) . Meanwhile , the collaborative filtering ( CF ) approach does not require product information but only relies on users ’ interaction history which can be either explicit or implicit feedback . CF systems can be divided into two major categories : i ) neighborhood-based models which suggest items that are most similar to the item that a user is interested in ( Herlocker et al. , 2000 ; Tintarev & Masthoff , 2007 ) , and ii ) matrix factorization models which could explore the latent factors connecting items to users in order to make accurate recommendations ( Ricci et al. , 2015 ; Koren et al. , 2009 ; Koren , 2008 ) . However , it is often the case that there is not enough transaction data to make accurate recommendations for a new user or item . To tackle this so-called cold-start problem , hybrid methods are proposed by combining auxiliary information into CF models ( Singh & Gordon , 2008 ; Agarwal & Chen , 2009 ; Wang & Blei , 2011 ; Li et al. , 2011 ; Rendle , 2010 ) . In the interest of the hybrid approach and its advantages , our study attempts to improve typical neighborhood-based RSs utilizing available content-related knowledge . The main contributions of this work are summarized as follows : • Introducing new methods to represent user preference via combining user ’ s interaction data and item ’ s content-based information , which helps to estimate the similarity between a user and an item . • Integrating the user-item similarity degree into the baseline estimate of neighborhood-based RSs to provide more precise recommendations , surpassing competitive hybrid models . The remainder of this paper is organized as follows . Section 2 reviews the basic knowledge on neighborhood-based CF systems , including hybrid models . Detail descriptions of our proposed methods are presented in Section 4 . Section 5 gives experimental results and in-depth analysis . At last , we conclude this study in Section 6 . 2 PRELIMINARIES . In this paper , u , v denote users and i , j denote items . rui denotes the preference by user u for item i , also known as the rating , where high values indicate strong preference , and all the ( u , i ) pairs are stored in the set K = { ( u , i ) |rui is known } . Meanwhile , R ( u ) denotes the set of all items rated by user u . In rating prediction task , the objective is to predict unknown rating r̂ui where user u has not rated item i yet . Popular neighborhood-based CF techniques for the rating prediction task and an existing hybrid variant are briefly reviewed as follows . 2.1 NEIGHBORHOOD-BASED MODELS . The neighborhood-based approach is one of the most popular techniques in CF , which is only based on the similarity between users or items to give recommendations . There are currently two methods for implementing neighborhood-based CF models : i ) user-oriented ( or user-user ) model which predicts a user ’ s preference based on similar users , and ii ) item-oriented ( or item-item ) model which finds similar items to the item a user liked and recommends these items to her . Of the two methods , the latter introduced by Sarwar et al . ( 2001 ) has become dominant . This is due to the fact that the number of users in real-life systems is often orders of magnitude bigger than of items , which makes the user-oriented model inefficient . Furthermore , the item-oriented model is capable of providing a rational explanation for recommendations ( Ricci et al. , 2015 ) . Therefore , our implementations in this work adopt the item-item approach as the base model . The fundamental of neighborhood-based models is similarity measure . By computing the similarity degree sij between all pairs of items i and j using popular similarity measures such as Cosine similarity ( Cos ) or Pearson Correlation Coefficients ( PCC ) , we can identify the set of k neighbors Sk ( i , u ) which consists of k most similar items to i rated by user u . Then , r̂ui can be predicted as a weighted average of the ratings of similar items : r̂ui = ∑ j∈Sk ( i , u ) sijruj∑ j∈Sk ( i , u ) sij ( 1 ) Even though Equation ( 1 ) can capture the user-item interactions , much of the observed ratings are due to the bias effects associated with either users or items , independently of their interactions . In detail , some items usually receive higher ratings than others , and some users tend to give higher ratings than others . kNNBaseline model proposed by Koren ( 2010 ) adjusts the above formula through a baseline estimate which accounts for the user and item effects as follows . r̂kNNBaselineui = bui + ∑ j∈Sk ( i ; u ) sij ( ruj − buj ) ∑ j∈Sk ( i , u ) sij ( 2 ) where bui = µ + bu + bi denotes the baseline estimate , µ denotes the mean of overall ratings , bu and bi correspond to the bias of user u and item i , respectively , which can be trained using popular optimization algorithms such as Stochastic Gradient Descent ( SGD ) or Alternating Least Squares ( ALS ) . 2.2 INTEGRATING CONTENT-BASED INFORMATION INTO NEIGHBORHOOD-BASED MODELS . A number of problems regarding kNN models using similarity measure on the rating information were noticed by Duong et al . ( 2019 ) . The first problem is the sparsity of the rating matrix , which might yield an inaccurate similarity score between two items that share only a few common users . Secondly , filtering common users who rated both items to calculate the similarity score is a timeconsuming task due to a large number of users . To address these problems , a novel similarity measure was proposed using item content-based information . Assuming that each item i is characterized by a feature vector qi = { qi1 , qi2 , ... , qif } ∈ Rf where f is the number of features , which is stored in matrix Q ∈ Rn×f . The value of each element encodes how strong an item exhibits particular properties . The similarity score sij between movies i and j is calculated as follows . sCos content ij = ∑f k=1 qikqjk√∑f k=1 q 2 ik √∑f k=1 q 2 jk ( 3 ) or sPCC content ij = ∑f k=1 ( qik − q̄i ) ( qjk − q̄j ) √∑f k=1 ( qik − q̄i ) 2 √∑f k=1 ( qjk − q̄j ) 2 ( 4 ) where q̄i and q̄j are the mean of feature vectors qi and qj , respectively . Experiments showed that the item-oriented CF models using Coscontent and PCCcontent provide equivalent accuracy to the stateof-the-art CF models using rating information whilst performing at least 2 times faster . Hereafter , kNNBaseline model using one of these similarity measures is referred to as kNNContent . 3 EXPERIMENTAL SETTING . 3.1 MOVIELENS DATASET AND EVALUATION CRITERIA . In this work , the MovieLens 20M dataset is used as a benchmark . This is a widely used dataset in the study of RSs which originally contains 20,000,263 ratings and 465,564 tag applications across 27,278 movies . The ratings are float values ranging from 0.5 to 5.0 with a step of 0.5 . Tag Genome data , which is computed on user-contributed content including tags , ratings , and textual reviews , is firstly introduced in this version of MovieLens dataset ( Harper & Konstan , 2016 ) . To utilize this kind of data , a cleaning process is applied to the dataset . Specifically , the movies without genome tags are excluded from the dataset . Then , only movies and users with at least 20 ratings are kept . Table 1 summarizes the result of the cleaning stage . The preprocessed dataset is split into 2 distinct parts : 80 % as the training set and the remaining 20 % as the testing set . To evaluate the performance of the proposed models , three commonly used indicators in the field of rating prediction are used : RMSE ( Root Mean Squared Error ) and MAE ( Mean Absolute Error ) for accuracy evaluation where smaller values indicate better performance , and Time [ s ] for timing evaluation . Here , RMSE and MAE are defined as follows . RMSE = √ ∑ u , i∈TestSet ( r̂ui − rui ) 2 /|TestSet| ( 5 ) MAE = ∑ u , i∈TestSet |r̂ui − rui| /|TestSet| ( 6 ) where |TestSet| is the size of the testing set . The total duration of the model ’ s learning process on the training set and predicting all samples in the testing set is measured as Time [ s ] . All experiments are carried out using Google Colaboratory with 25GB RAM and no GPU . 3.2 BASELINE MODELS . In this paper , several popular models are selected as baselines to evaluate the proposed methods . Firstly , we implement two competitive neighborhood-based models including kNNBaseline ( Koren , 2010 ) and kNNContent ( Duong et al. , 2019 ) . Besides , SVD ( Funk , 2006 ) and SVD++ ( Koren , 2008 ) , two well-known representatives of matrix factorization technique , are also experimented due to their superior accuracy and flexible scalability with extremely sparse data ( Funk , 2006 ) . Error rates and the time to make predictions are measured for comparison . The optimal hyperparameters for each baseline model are carefully chosen using 5-fold cross-validation . In particular , the error rates of the neighborhood-based models are calculated with the neighborhood size k ∈ { 10 , 15 , 20 , 25 , 30 , 35 , 40 } . Due to better performance compared to Cos and Coscontent , PCC and PCCcontent are chosen as the similarity measures in kNNBaseline and kNNContent models . SVD and SVD++ models are trained using 40 hidden factors with 100 iterations and the step size of 0.002 .

Summary: In this paper, the authors study the recommendation systems. They first propose three new methods to characterize user preferences utilizing both rating data and item content information. The first method takes the weighted average of the feature vectors of items, the second one incorporates the effect of biases in the first one, and the third method puts weight on the biased feature vector of users using the item feature vector.  Next, they revise the baseline estimate by integrating the user-item similarity. They further integrate these methods into two baseline models, namely kNNBaseline and kNNContent. Experimental results suggest that these improvements lead to an increase in accuracy (2%-6%) with the expense of the additional complexity.