Collaborative Generated Hashing for Market Analysis and Fast Cold-start Recommendation

1 INTRODUCTION . With the explosion of e-commerce , most customers are accustomed to receiving a variety of recommendations , such as movies , books , news , or hotels they might be interested in . Traditional recommender systems just recommended items that are similar to what they liked or rated in the previous . Recommendations help users find their desirable items , and also creates new revenue opportunities for vendors , such as Amazon , Taobao , eBay , etc . Among them , one of the most popular recommendation methods , collaborative filtering is dependent on a large amount of user-item interactive information to provide an accurate recommendation . However , most of new e-commerce vendors do not have enough interactive data , which leads to low recommendation accuracy , i.e. , cold-start issues . Previous studies on cold-start issues generally modeled as a combination of collaborative filtering and content filtering , known as hybrid recommender systems . Specifically , they learned real latent factors by incorporating the side information into the interactive data . Such as Collaborative Deep Learning ( CDL ) ( Wang et al. , 2015 ) , Visual Bayesian Personalized Ranking ( VBPR ) ( He & McAuley , 2016 ) , Collaborative Topic modeling for Recommedation ( CTR ) ( Wang & Blei , 2011 ) , and the DropoutNet for addressing cold start ( DropoutNet ) ( Volkovs et al. , 2017 ) , ABCPRec for Bridging Consumer and Producer Roles for User-Generated Content Recommendation ( ABCPRec ) ( Tsukuda et al. , 2019 ) . All of the above hybrid recommender systems were modeled in real latent space , which leads to low efficiency for the online recommendation with the increasing scale of datasets . Recent studies show the promising of hashing based methods to tackle the efficiency challenge by representing users and items with binary codes ( Zhang et al. , 2014 ; Zhou & Zha , 2012 ; Zhang et al. , 2016 ; Liu et al. , 2019 ) , because the preference score can be represented by the Hamming distance calculated via XOR operation efficient ( Wang et al. , 2014 ) . However , the existing hashing based recommendations are learning-based frameworks , which leads to NP-hard problems of optimizing discrete objectives . Thus many scholars learned binary codes by some approximate techniques , such as the two-stage hashing learning method utilized in Preference Preserving Hashing ( PPH ) ( Zhang et al. , 2014 ) and the Iterative Quantization ( ITQ ) ( Zhou & Zha , 2012 ) . To reduce information loss , two learning-based hashing frameworks : bit-wise learning and block-wise learning were respectively proposed in hashing based recommendation frameworks ( Zhang et al. , 2016 ; Wang et al. , 2019 ; Zhang et al. , 2018 ; Zheng et al. ) . However , due to the requirement of binary outputs for learning-based hashing frameworks , the training procedure is expensive for large-scale recommendation , which motivates us to propose a generative approach to learn hash functions . In this paper , we propose the collaborative generated hashing ( CGH ) to learn hash functions of users and items from content data with the principle of Minimum Description Length ( MDL ) ( Dai et al. , 2017 ) . In marketing area , mining potential customers is crucial to the e-commerce . CGH provides a strategy to discover potential users by the generative step . To reconstruct effective users , uncorrelated and balanced limits are imposed to learn compact and informative binary codes with the principle of the MDL . Especially , discovering potential customers is vital to the success of adding new items for a recommendation platform ( Papies et al. , 2017 ) . Specifically , for a new item , we can generate a new potential user by the generative step ( detailed in Section 2.1 ) , and then search the nearest potential users in the user set . By recommending a new product to the potential users who might be interested in but didn ’ t plan to buy , further e-commerce strategies can be developed to attract those potential users . We organize the paper as follows : Section 2 introduce the main techniques of CGH . We first introduce the framework of CGH and compare it with the closely related competing baselines : CDL ( Wang et al. , 2015 ) and DropoutNet ( Volkovs et al. , 2017 ) ; we then formulate the generative step in Section 2.1 and the inference step in Section 2.2 , respectively ; we finally summarize the training objective and introduce the optimization in Section 2.3 . Particularly , we demonstrate the process of mining potential users for the marketing application in Section 2.1 . Section 3 presents the experimental results for marketing analysis and recommendation accuracy in various settings . Section 4 concludes the paper . The main contributions of this paper are summarized as follows : ( 1 ) We propose the Collaborative Generated Hashing ( CGH ) with the principle of MDL to learn compact but informative hash codes , which applies to various settings for recommendation . ( 2 ) We provides a marketing strategy by discovering potential users by the generative step of CGH , which can be applied to boost the e-commence development . ( 3 ) We evaluate the effectiveness of the proposed CGH compared with the state-of-the-art baselines , and demonstrate its robustness and convergence properties on the public datasets . 2 COLLABORATIVE GENERATED HASHING . The framework of the proposed CGH is shown in Fig . 1 ( c ) , where U , V and R are respectively observed user content , item content and rating matrix . B and D are binary codes of users and items , respectively . CGH consists of the generative step marked as dashed lines and the inference step denoted by solid lines . Once training is finished , we fix the model and make forward passes to obtain binary codes B and D through the inference step , and then conduct recommendation . For the marketing application , we create a new user via the generative step . In comparison of CGH with the closely related baseline CDL ( Wang et al. , 2015 ) , the proposed CGH aims to learn binary codes instead of real latent vectors P and Q due to the advantage of hashing for online recommendation ; plus the CGH optimizes an objective with the principle of MDL , while CDL optimized the joint objective of rating loss and item content reconstruction error . In comparison of CGH with DropoutNet ( Volkovs et al. , 2017 ) , CGH can be used as a marketing strategy by discovering potential users ; plus CGH learns hash functions by stacked denoising autoendoer , while DropoutNet obtained real latent factors by the standard neural network . In the following we start by first formulating the generative process and demonstrating the application in marketing area ; we then formulate the inference step ; we finally summarize the training objective and the optimization method . 2.1 MINING POTENTIAL USERS. . Give a sparse rating matrix R and item content data V ∈ Rdv , where dv is the dimension of the content vector , and V is stacked by the bag-of-words vectors of item content in the item set V . most previous studies were focus on modeling deterministic frameworks to learn representations of items for item recommendation , such as CDL , CTR , DropoutNet , et.al . In this paper , we discover a new strategy from a perspective of marketing for item recommendation – mining potential users . We demonstrate the process of mining potential users by an item through the generative step in Fig . 2 . After the inference step , the binary code of item j is available . By maximizing the similarity function δ ( bi , dj ) ( detailed in Section 2.1 ) , the optimal binary code bp is obtained . Then we generate the new user up through the generative step . Finally we find out potential users from the user set by some nearest neighborhood algorithms , such as KNN . As a marketing strategy , it can discover potential users for both warm-start items and cold-start items . Thus , from a perspective of marketing , it can be regarded as another kind of item recommendation . The generation process ( also referred as decoding process ) is denoted by dashed lines in Fig . 1 ( c ) . Fix binary codes bi and dj of the user i and the item j , the bag-of-words vector ui of the user i ( vj of the item j ) is generated via p ( θu ) . The ratings rij is generated by bi and dj . We use a simple Gaussian distribution to model the generation of ui and vj given bi and dj like Stochastic Generative Hashing ( SGH ) ( Dai et al. , 2017 ) : p ( ui|bi ) = N ( Cubi , λu-1I ) , p ( vj |dj ) = N ( Cvdj , λv -1I ) , ( 1 ) where Cuk = [ cuk ] rk=1 , cuk ∈ Rdu is the codebook ( Dai et al. , 2017 ) with r codewords , which is similar for Cv , and du is the dimension of the bag-of-the-words vector of users . The prior is modeled as the multivariate Bernoulli distribution on hash codes : p ( bi ) ∼ B ( ρu ) , and p ( dj ) ∼ B ( ρv ) , thus the prior probability are as follows : p ( bi ) = ∏r k=1 ρbikui ( 1− ρui ) 1−bik , p ( dj ) = ∏r k=1 ρ djk vj ( 1− ρvj ) 1−djk , ( 2 ) We formulate the rating with the similarity between binary codes of users and items like the most successful recommender systems , matrix factorization ( Koren et al. , 2009 ) . Then the rating is thus drawn from the normal distribution centered at the similarity value , p ( rij |bi , dj ) ∼ N ( δ ( bi , dj ) , C−1ij ) , ( 3 ) where δ ( bi , dj ) = 1 − 1rHamdis ( bi , dj ) denotes the similarity between binary codes bi and dj . Hamdis ( bi , dj ) represents Hamming distance between the two binary vectors , which has been widely applied in hashing-based recommender system ( Wang & Blei , 2011 ; Lian et al. , 2017 ; Zhang et al. , 2018 ) . Cij is the precision parameter that serves as confidence for rij similar to that in CTR ( Wang & Blei , 2011 ) ( Cij = a if rij = 1 and Cij = b otherwise ) due to the fact that rij = 0 means the user i is either not interested in item j or not aware of it . With the generative model constructed , the joint probability of both observed ratings , content vectors and binary codes is given by p ( R , U , V , B , D ) = ∏ i , j p ( rij |bi , dj ) p ( ui|bi ) p ( vj |dj ) p ( bi ) p ( dj ) ( 4 ) 2.2 CONSTRAINTS ON BINARY LATENT VARIABLES . The inference process ( also referred as encoding process ) shown in the Fig . 1 ( c ) with dashed lines , the binary latent variables bi ( dj ) depends on the content vector ui ( vj ) and the ratingR ( shadowed in Fig 1 ) . Inspired by the recent work on generative hashing ( Dai et al. , 2017 ) and DropoutNet ( Volkovs et al. , 2017 ) , we use a multivariate Bernoulli distribution to model the inference process of bi and dj with linear parametrization , i.e. , q ( bi|ũi ) = B ( σ ( TuT ũi ) ) q ( dj |ṽj ) = B ( σ ( TvT ṽj ) ) , ( 5 ) where ũi = [ ui , pi ] , ṽj = [ vj , qj ] . pi and qj are the results of r-dimension matrix factorization ( Koren et al. , 2009 ) of R , i.e. , rij ≈ pTi qj . Tu = [ tuk ] rk=1 , tuk ∈ Rdu+r , Tv = [ tvk ] rk=1 , tvk ∈ Rdv+r are the transformation matrices of linear parametrization . From SGH ( Dai et al. , 2017 ) , the MAP solution of the eq . ( 5 ) is readily given by bi = argmax bi q ( bi|ui ) = sign ( T Tu ui ) + 1 2 , dj = argmax dj q ( dj |vj ) = sign ( T Tv vj ) + 1 2 . ( 6 ) With the linear projection followed by a sign function , we can easily get hash codes of users and items . However , hashing with a simple sign function suffers from large information loss according to ( Zhang et al. , 2016 ) , which motivates us to add constraints on parameters in the inference step . To derive compact and informative hash codes for users and items , we add balanced and uncorrelated constraints in the inference step . The balanced constraint is proposed to maximize the entropy of each binary bit ( Zhou & Zha , 2012 ) , and the uncorrelated constraint makes each bit is independent of others . Then we can obtain compact and informative hash codes by the following constraints , Balanced constraint : ∑ k bik = 0 , ∑ k djk = 0 Uncorrelated constraint : bibTi = Ir , djd T j = Ir ( 7 ) From the eq . ( 6 ) , bi and dj are only dependent on parameters Tu and Tv , respectively , thus we add constraints on Tu and Tv directly . So eq . ( 7 ) is equivalent to the following constraints , Balanced constraint : T Tu 1r = 0 , T Tv 1r = 0 , Uncorrelated constraint : T Tu Tu = Idu+r , T Tv Tv = Idv+r . ( 8 ) By imposing the above constraints in the training step , compact and informative hash codes can be obtained through the inference process . Next we summarize the training objective and its optimization .

Summary: The work considers the problem of efficient user and item recommendations in the warm- and cold-start settings. It aims at improving computational efficiency of the best candidate selection in these settings by utilizing binary codes representation. The transformation from an actual to a binary code representation is learned in a hybrid manner using both collaborative and content information. In order to keep such representations compact yet expressive enough, the authors impose a set of constraints that ensure balanced and uncorrelated transformations. Once binary codes are learned, the inference can be made by virtue of efficient Hamming distance computations. Moreover, the search for candidate entities can be performed via the generative step that projects binary codes onto actual feature space, where kNN-based techniques can be further utilized.