Untangle: Critiquing Disentangled Recommendations

1 Introduction . Figure 1 : Untangle model is trained in two phases : Disentangling phase : Input to encoder is a one hot representation of an item ( green dotted line ) . Obtained representation is disentangled across A attributes . Recommendation phase : Input to encoder is the items user interacted with ( solid red line ) and recommends new items . User and item representations form the basis of typical collaborative filtering recommendation models . These representations can be learned through various techniques such as Matrix Factorization ( 1 ; 2 ) , or are constructed dynamically during inference e.g . the hidden state of RNN ’ s in session-based recommendations ( 3 ; 4 ) . As most standard recommendation models solely aim at increasing the performance of the system , no special care is taken to ensure interpretability of the user and item representations . These representations do not explicitly encode user preferences over item attributes . Hence , they can not be easily used by users to change a.k.a . critique ( 5 ) the recommendations . For instance , a user in a recipe recommendation system can not ask for recommendations for a set of less spicy recipes , as the spiciness is not explicitly encoded in the latent space . Moreover the explainability of the recommendations that are provided by such systems is very limited . In this work , we enrich a state-of-the-art recommendation model to explicitly encode preferences over item attributes in the user latent space while simultaneously optimizing for recommendation ’ s performance . Our work is motivated by disentangled representations in other domains , e.g. , manipulating generative models of images with specific characteristics ( 6 ) or text with certain attributes ( 7 ) . Variational Autoencoders ( VAEs ) , particularly β-VAE ’ s ( 8 ) ( which we adapt here ) , are generally used to learn these disentangled representations . Intuitively , they optimize embeddings to capture meaningful aspects of users and items independently . Consequently , such embeddings will be more usable for critiquing . There are two types of disentangling β-VAEs : unsupervised and supervised . In the former , the representations are disentangled to explanatory factors of variation in an unsupervised manner , i.e. , without assuming additional information on the existence ( or not ) of specific aspects . Used in the original β-VAE ( 8 ) approach , a lack of supervision often results in inconsistency and instability in disentangled representations ( 9 ) . In contrast , in supervised disentangling , a small subset of data is assumed to have side-information ( i.e . a label or a tag ) . This small subset is then used to disentangle into meaningful factors ( 10 ; 9 ) . As critiquing requires user control using familiar terms/attributes , we incorporate supervised disentanglement in a β-VAE architecture in this work . To achieve the explicit encoding of preferences over item attributes in embedding space we refine the training strategy of the untangle model . We essentially train in two phases : i ) Disentangling phase : We explicitly disentangle item representations , using very few supervised labels . ii ) Recommendation phase : We encode the user , using the bag-of-words representation of the items interacted , and then generate the list of recommended items . Untangle gives fine-grained control over the recommendations across various item attributes , as compared to the baseline . We achieve this with a tiny fraction of attribute labels over items , and moreover achieve comparable recommendation performance compared to state-of-the-art baselines . 2 Related Work . Deep learning based Autoencoder architectures are routinely used in collaborative filtering and recommendation models ( 11 ; 12 ; 13 ) . In particular ( 11 ; 12 ) adopt denoising autoencoder architectures , whereas ( 13 ) uses variational autoencoders . The internal ( hidden ) representations generated by the encoders in these models are not interpretable and hence can not be used for critiquing or explanations in recommendations . Recent work on Variational Autoencoders across domains have focused on the task of generating disentangled representations . One of the first approaches used to that end was β-VAE ( 8 ; 14 ; 15 ) , which essentially enforced a stronger ( multiplying that term with β > 1 ) KL divergence constraint on the VAE objective . Such representations are more controllable and interpretable as compared to VAEs . One of the drawbacks of β-VAE is that the disentanglement of the factors can not be controlled and that they are relatively unstable and not easy to reproduce particularly when the factors of variance are subtle ( 9 ; 8 ; 14 ; 16 ; 17 ) . This has motivated methods that explicitly supervise the disentangling ( 10 ) , that rely either on selecting a good set of disentangling using multiple runs and the label information ( 18 ) , or by adding a supervised loss function in the β-VAE objective function ( 10 ) . As supervised disentangling methods are better in explainability and could provide control over desired attributes , we motivate our model from ( 19 ) for better critiquing in VAE based recommendation systems . In recommender systems similar methods to utilize side information , have also been used recently to allow for models that enable critiquing of recommendations . These models allow users to tune the recommendations across some provided attributes/dimensions . Notable examples are ( 20 ; 21 ) , where the models are augmented with a classifier of the features over which to control the recommendation . Adjusting the features at the output of the classifier modifies the internal hidden state of the model and leads to recommendations that exhibit or not the requested attribute . Note that this method of critiquing is quite different to our approach which allows for a gradual adjustment of the attributtes . Moreover the models in ( 20 ; 21 ) require a fully labeled dataset with respect to the attributes while our approach only requires a small fraction of labeled data . Unsupervised disentanglement was also recently used to identify and potentially use factors of variation from purely collaborative data i.e. , data generated by user interactions with items ( 22 ) note though that this method focus was mainly on performance of the recommendations and that it does not allow for seamless critiquing as it is not clear what aspect of the data get disentangled . 3 Untangle . The aim of the untangle model is to obtaining controllable user ( and item ) representations for better critiquing along with optimizing for recommendation performance . To this end , we incorporate a simple supervised disentanglement technique to disentangle across item attributes/characteristics over which we want to provide explicit control to the users . We index users with u ∈ { 1 , . . . , n } , and items with i ∈ { 1 , . . . , m } . Xn×m is a matrix of user-item interactions ( xui = 1 if user u interacted with item i , and 0 otherwise ) . A subset of items are assumed to have binary labels for attributes A . Our model is a modified β-VAE architecture , with a feed forward network based encoder and decoder . In Figure 1 , user u is represented by [ z : c ] . Note that : stands for concatenation , the z part of the representation is non-interpretable by default while on the c part of the representation we map ( through a refined learning step ) the representation of the attributes of the items over which we would like the user to have control . Each dimension in c is mapped to only one attribute a . Across the paper , we refer the dimension associated with the attribute a , as ca . The user representation is sampled from the distribution parameterized by the encoder ( qφ ) : qφ ( xu∗ ) = N ( µφ ( xu∗ ) , diag ( σφ ( xu∗ ) ) . The input to the encoder is the bag of words representation of the items u interacted with , i.e . the uth row of matrix X , xu∗ . The decoder generates the probability distribution given user representation [ z : c ] , π ( z ) ∝ exp ( fdecφ ( [ z : c ] ) ) , over the m items . The likelihood function used in recommender system settings ( 3 ; 23 ; 24 ; 25 ) is typically the multinomial likelihood : pθ ( xu| [ z : c ] ) = ∑ i xui log πi ( [ z : c ] ) ) 3.1 Learning . Training is conducted in two phases : Recommendation and Disentangle phase , as mentioned in Algorithm 1 . Recommendation Phase The objective in this phase is to optimize the encoder parameterized by ( θ ) , and decoder parameterized by ( ψ ) to generate personalized recommendations . We train our model with the following objective : L ( xu∗ , θ , φ ) ≡ Eqθ ( [ z : c ] |xu∗ ) [ logpθ ( xu∗| [ z : c ] ) ] − βKL ( qφ ( [ z : c ] |xu∗ ) |p ( [ z : c ] ) ) ( 1 ) Intuitively , this is the negative reconstruction error minus the Kullback-Leibler divergence enforcing the posterior distribution of z to be close to the Gaussian distribution ( prior ) p ( z ) . The KL divergence in β-VAE is computed between the representation sampled from the encoder and the normal distribution p ( z ) = N ( 0 , Id ) . The diagonal co-variance matrix enforces a degree of independence among the individual factors of the representation . Consequently , increasing the weight of the KL divergence term with β > 1 boosts the feature independence criteria , leading to disentangled representation . This ensures that even in the recommendation phase , the learnt user representations are nudged towards disentanglement . Disentanglement Phase Since the attribute information is commonly available across the items . In this phase , we first obtain the item representation in the user latent space ( as depicted in the highlighted green box in Figure 1 ) . We pass the one hot encoding of an item , and obtain its representation in the latent user space . We then disentangle the obtained representation using the following objective : ( 2 ) L ( 1i , θ , φ ) ≡ Eqθ ( [ z : c ] |1i ) [ logpθ ( 1i| [ z : c ] ) ] − βKL ( qφ ( [ z : c ] |1i ) |p ( [ z : c ] ) ) + γEqθ ( c|1i ) l ( qφ ( c|1i ) , a ) Algorithm 1 : Untangle : Training Data : X ∈ Rn×m containing user-item interactions , with a subset of items having labels for A attributes 1 initialize model params . : Encoder ( φ ) , Decoder ( θ ) ; 2 do 3 if is_disentangle then // Disentangle representations 4 1i ← random mini batch from set of items that are labelled with A set . 5 [ z : c ] ← sample from N ( µφ ( 1i ) , diag ( σφ ( 1i ) ) 6 x̃i∗ ← Decoder ( [ z : c ] ) 7 compute gradients ∇Lφ , ∇Lθ using Objective 2 8 φ← φ+∇Lφ 9 θ ← θ +∇Lθ 10 end 11 if is_recommend then // Recommend items 12 xu∗ ← random mini-batch from dataset 13 [ z : c ] ← sample from N ( µφ ( xu∗ ) , diag ( σφ ( xu∗ ) ) 14 x̃u∗ ← Decoder ( [ z : c ] ) 15 compute gradients ∇Lφ , ∇Lθ using Objective 1 16 φ← φ+∇Lφ 17 θ ← θ +∇Lθ 18 end 19 while model converges ; As in ( 10 ) , we modify the β-VAE objective ( Objective 1 ) in to incorporate a classification loss over the factors c , over which we disentangle . This loss penalizes discrepancies between the attribute label prediction for factor ca and the label a of interest , nudging the disentanglement for each attribute to happen over the corresponding factor ca .

Summary: The submission aims to develop a disentangled recommender that explicitly grants the users the ability to control a chosen aspect of the recommendation list. This is achieved by (1) learning disentangled user/item representations via beta-VAE and (2) aligning text tags with certain dimensions via an auxiliary loss (the term after gamma in Eq 2). Empirical results demonstrate that this supervised approach outperforms the unsupervised baselines in terms of disentanglement and controllability, e.g., it can explicitly control the recommender to recommend a list of romantic movies.