Collaborative Generated Hashing for Market Analysis and Fast Cold-start Recommendation

1 INTRODUCTION . With the explosion of e-commerce , most customers are accustomed to receiving a variety of recommendations , such as movies , books , news , or hotels they might be interested in . Traditional recommender systems just recommended items that are similar to what they liked or rated in the previous . Recommendations help users find their desirable items , and also creates new revenue opportunities for vendors , such as Amazon , Taobao , eBay , etc . Among them , one of the most popular recommendation methods , collaborative filtering is dependent on a large amount of user-item interactive information to provide an accurate recommendation . However , most of new e-commerce vendors do not have enough interactive data , which leads to low recommendation accuracy , i.e. , cold-start issues . Previous studies on cold-start issues generally modeled as a combination of collaborative filtering and content filtering , known as hybrid recommender systems . Specifically , they learned real latent factors by incorporating the side information into the interactive data . Such as Collaborative Deep Learning ( CDL ) ( Wang et al. , 2015 ) , Visual Bayesian Personalized Ranking ( VBPR ) ( He & McAuley , 2016 ) , Collaborative Topic modeling for Recommedation ( CTR ) ( Wang & Blei , 2011 ) , and the DropoutNet for addressing cold start ( DropoutNet ) ( Volkovs et al. , 2017 ) , ABCPRec for Bridging Consumer and Producer Roles for User-Generated Content Recommendation ( ABCPRec ) ( Tsukuda et al. , 2019 ) . All of the above hybrid recommender systems were modeled in real latent space , which leads to low efficiency for the online recommendation with the increasing scale of datasets . Recent studies show the promising of hashing based methods to tackle the efficiency challenge by representing users and items with binary codes ( Zhang et al. , 2014 ; Zhou & Zha , 2012 ; Zhang et al. , 2016 ; Liu et al. , 2019 ) , because the preference score can be represented by the Hamming distance calculated via XOR operation efficient ( Wang et al. , 2014 ) . However , the existing hashing based recommendations are learning-based frameworks , which leads to NP-hard problems of optimizing discrete objectives . Thus many scholars learned binary codes by some approximate techniques , such as the two-stage hashing learning method utilized in Preference Preserving Hashing ( PPH ) ( Zhang et al. , 2014 ) and the Iterative Quantization ( ITQ ) ( Zhou & Zha , 2012 ) . To reduce information loss , two learning-based hashing frameworks : bit-wise learning and block-wise learning were respectively proposed in hashing based recommendation frameworks ( Zhang et al. , 2016 ; Wang et al. , 2019 ; Zhang et al. , 2018 ; Zheng et al. ) . However , due to the requirement of binary outputs for learning-based hashing frameworks , the training procedure is expensive for large-scale recommendation , which motivates us to propose a generative approach to learn hash functions . In this paper , we propose the collaborative generated hashing ( CGH ) to learn hash functions of users and items from content data with the principle of Minimum Description Length ( MDL ) ( Dai et al. , 2017 ) . In marketing area , mining potential customers is crucial to the e-commerce . CGH provides a strategy to discover potential users by the generative step . To reconstruct effective users , uncorrelated and balanced limits are imposed to learn compact and informative binary codes with the principle of the MDL . Especially , discovering potential customers is vital to the success of adding new items for a recommendation platform ( Papies et al. , 2017 ) . Specifically , for a new item , we can generate a new potential user by the generative step ( detailed in Section 2.1 ) , and then search the nearest potential users in the user set . By recommending a new product to the potential users who might be interested in but didn ’ t plan to buy , further e-commerce strategies can be developed to attract those potential users . We organize the paper as follows : Section 2 introduce the main techniques of CGH . We first introduce the framework of CGH and compare it with the closely related competing baselines : CDL ( Wang et al. , 2015 ) and DropoutNet ( Volkovs et al. , 2017 ) ; we then formulate the generative step in Section 2.1 and the inference step in Section 2.2 , respectively ; we finally summarize the training objective and introduce the optimization in Section 2.3 . Particularly , we demonstrate the process of mining potential users for the marketing application in Section 2.1 . Section 3 presents the experimental results for marketing analysis and recommendation accuracy in various settings . Section 4 concludes the paper . The main contributions of this paper are summarized as follows : ( 1 ) We propose the Collaborative Generated Hashing ( CGH ) with the principle of MDL to learn compact but informative hash codes , which applies to various settings for recommendation . ( 2 ) We provides a marketing strategy by discovering potential users by the generative step of CGH , which can be applied to boost the e-commence development . ( 3 ) We evaluate the effectiveness of the proposed CGH compared with the state-of-the-art baselines , and demonstrate its robustness and convergence properties on the public datasets . 2 COLLABORATIVE GENERATED HASHING . The framework of the proposed CGH is shown in Fig . 1 ( c ) , where U , V and R are respectively observed user content , item content and rating matrix . B and D are binary codes of users and items , respectively . CGH consists of the generative step marked as dashed lines and the inference step denoted by solid lines . Once training is finished , we fix the model and make forward passes to obtain binary codes B and D through the inference step , and then conduct recommendation . For the marketing application , we create a new user via the generative step . In comparison of CGH with the closely related baseline CDL ( Wang et al. , 2015 ) , the proposed CGH aims to learn binary codes instead of real latent vectors P and Q due to the advantage of hashing for online recommendation ; plus the CGH optimizes an objective with the principle of MDL , while CDL optimized the joint objective of rating loss and item content reconstruction error . In comparison of CGH with DropoutNet ( Volkovs et al. , 2017 ) , CGH can be used as a marketing strategy by discovering potential users ; plus CGH learns hash functions by stacked denoising autoendoer , while DropoutNet obtained real latent factors by the standard neural network . In the following we start by first formulating the generative process and demonstrating the application in marketing area ; we then formulate the inference step ; we finally summarize the training objective and the optimization method . 2.1 MINING POTENTIAL USERS. . Give a sparse rating matrix R and item content data V ∈ Rdv , where dv is the dimension of the content vector , and V is stacked by the bag-of-words vectors of item content in the item set V . most previous studies were focus on modeling deterministic frameworks to learn representations of items for item recommendation , such as CDL , CTR , DropoutNet , et.al . In this paper , we discover a new strategy from a perspective of marketing for item recommendation – mining potential users . We demonstrate the process of mining potential users by an item through the generative step in Fig . 2 . After the inference step , the binary code of item j is available . By maximizing the similarity function δ ( bi , dj ) ( detailed in Section 2.1 ) , the optimal binary code bp is obtained . Then we generate the new user up through the generative step . Finally we find out potential users from the user set by some nearest neighborhood algorithms , such as KNN . As a marketing strategy , it can discover potential users for both warm-start items and cold-start items . Thus , from a perspective of marketing , it can be regarded as another kind of item recommendation . The generation process ( also referred as decoding process ) is denoted by dashed lines in Fig . 1 ( c ) . Fix binary codes bi and dj of the user i and the item j , the bag-of-words vector ui of the user i ( vj of the item j ) is generated via p ( θu ) . The ratings rij is generated by bi and dj . We use a simple Gaussian distribution to model the generation of ui and vj given bi and dj like Stochastic Generative Hashing ( SGH ) ( Dai et al. , 2017 ) : p ( ui|bi ) = N ( Cubi , λu-1I ) , p ( vj |dj ) = N ( Cvdj , λv -1I ) , ( 1 ) where Cuk = [ cuk ] rk=1 , cuk ∈ Rdu is the codebook ( Dai et al. , 2017 ) with r codewords , which is similar for Cv , and du is the dimension of the bag-of-the-words vector of users . The prior is modeled as the multivariate Bernoulli distribution on hash codes : p ( bi ) ∼ B ( ρu ) , and p ( dj ) ∼ B ( ρv ) , thus the prior probability are as follows : p ( bi ) = ∏r k=1 ρbikui ( 1− ρui ) 1−bik , p ( dj ) = ∏r k=1 ρ djk vj ( 1− ρvj ) 1−djk , ( 2 ) We formulate the rating with the similarity between binary codes of users and items like the most successful recommender systems , matrix factorization ( Koren et al. , 2009 ) . Then the rating is thus drawn from the normal distribution centered at the similarity value , p ( rij |bi , dj ) ∼ N ( δ ( bi , dj ) , C−1ij ) , ( 3 ) where δ ( bi , dj ) = 1 − 1rHamdis ( bi , dj ) denotes the similarity between binary codes bi and dj . Hamdis ( bi , dj ) represents Hamming distance between the two binary vectors , which has been widely applied in hashing-based recommender system ( Wang & Blei , 2011 ; Lian et al. , 2017 ; Zhang et al. , 2018 ) . Cij is the precision parameter that serves as confidence for rij similar to that in CTR ( Wang & Blei , 2011 ) ( Cij = a if rij = 1 and Cij = b otherwise ) due to the fact that rij = 0 means the user i is either not interested in item j or not aware of it . With the generative model constructed , the joint probability of both observed ratings , content vectors and binary codes is given by p ( R , U , V , B , D ) = ∏ i , j p ( rij |bi , dj ) p ( ui|bi ) p ( vj |dj ) p ( bi ) p ( dj ) ( 4 ) 2.2 CONSTRAINTS ON BINARY LATENT VARIABLES . The inference process ( also referred as encoding process ) shown in the Fig . 1 ( c ) with dashed lines , the binary latent variables bi ( dj ) depends on the content vector ui ( vj ) and the ratingR ( shadowed in Fig 1 ) . Inspired by the recent work on generative hashing ( Dai et al. , 2017 ) and DropoutNet ( Volkovs et al. , 2017 ) , we use a multivariate Bernoulli distribution to model the inference process of bi and dj with linear parametrization , i.e. , q ( bi|ũi ) = B ( σ ( TuT ũi ) ) q ( dj |ṽj ) = B ( σ ( TvT ṽj ) ) , ( 5 ) where ũi = [ ui , pi ] , ṽj = [ vj , qj ] . pi and qj are the results of r-dimension matrix factorization ( Koren et al. , 2009 ) of R , i.e. , rij ≈ pTi qj . Tu = [ tuk ] rk=1 , tuk ∈ Rdu+r , Tv = [ tvk ] rk=1 , tvk ∈ Rdv+r are the transformation matrices of linear parametrization . From SGH ( Dai et al. , 2017 ) , the MAP solution of the eq . ( 5 ) is readily given by bi = argmax bi q ( bi|ui ) = sign ( T Tu ui ) + 1 2 , dj = argmax dj q ( dj |vj ) = sign ( T Tv vj ) + 1 2 . ( 6 ) With the linear projection followed by a sign function , we can easily get hash codes of users and items . However , hashing with a simple sign function suffers from large information loss according to ( Zhang et al. , 2016 ) , which motivates us to add constraints on parameters in the inference step . To derive compact and informative hash codes for users and items , we add balanced and uncorrelated constraints in the inference step . The balanced constraint is proposed to maximize the entropy of each binary bit ( Zhou & Zha , 2012 ) , and the uncorrelated constraint makes each bit is independent of others . Then we can obtain compact and informative hash codes by the following constraints , Balanced constraint : ∑ k bik = 0 , ∑ k djk = 0 Uncorrelated constraint : bibTi = Ir , djd T j = Ir ( 7 ) From the eq . ( 6 ) , bi and dj are only dependent on parameters Tu and Tv , respectively , thus we add constraints on Tu and Tv directly . So eq . ( 7 ) is equivalent to the following constraints , Balanced constraint : T Tu 1r = 0 , T Tv 1r = 0 , Uncorrelated constraint : T Tu Tu = Idu+r , T Tv Tv = Idv+r . ( 8 ) By imposing the above constraints in the training step , compact and informative hash codes can be obtained through the inference process . Next we summarize the training objective and its optimization .

Summary: This paper introduces a collaborative generated hashing (CGH) method to learn hash funcations of users and items from content data. The approach first provides a strategy to discover potential users by the generative step and inference through adding balanced and uncorrelated constraints. The experiments demonstrates some effectiveness on improving accuracy for both warm-start and cold-start recommendations.