Graph Neural News Recommendation with Unsupervised Preference Disentanglement (2020)

Abstract: With the explosion of news information, personalized news recommendation has become very important for users to quickly find their interested contents. Most existing methods usually learn the representations of users and news from news contents for recommendation. However, they seldom consider high-order connectivity underlying the user-news interactions. Moreover, existing methods failed to disentangle a user’s latent preference factors which cause her clicks on different news. In this paper, we model the user-news interactions as a bipartite graph and propose a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement, named GNUD. Our model can encode high-order relationships into user and news representations by information propagation along the graph. Furthermore, the learned representations are disentangled with latent preference factors by a neighborhood routing algorithm, which can enhance expressiveness and interpretability. A preference regularizer is also designed to force each disentangled subspace to independently reflect an isolated preference, improving the quality of the disentangled representations. Experimental results on real-world news datasets demonstrate that our proposed model can effectively improve the performance of news recommendation and outperform state-of-the-art news recommendation methods.

## introduction the amount of news and articles on many news platforms, such as google news 1 , has been growing constantly at an explosive rate, making it difficult for users to seek for news that they are interested in. in order to tackle the problem of information overload and meet the needs of users, news recommendation has been playing an increasingly important role for mining users' reading interest and providing personalized contents @xcite . a core problem in news recommendation is how to learn better representations of users and news. recently, many deep learning based methods have been proposed to automatically learn informative user and news representations @xcite . for instance, dkn @xcite learns knowledge-aware news representation via multi-channel cnn and gets a representation of a user by aggregating her clicked news history with different weights. however, these methods @xcite @xcite usually focus on news contents, and seldom consider the collaborative signal in the form of high-order connectivity underlying the user-news interactions. capturing high-order connectivity among users and news could deeply exploit structure characteristics and alleviate the sparsity, thus improving the rec-ommendation performance @xcite . for example, as shown in figure 1 , the high-order relationship u 1 -d 1 -u 2 indicates the behavior similarity between u 1 and u 2 so that we may recommend d 3 to u 2 since u 1 clicked d 3 , while d 1 -u 2 -d 4 implies d 1 and d 4 may have similar target users. moreover, users may click different news due to their great diversity of preferences. the real-world user-news interactions arise from highly complex latent preference factors. for example, as shown in figure 1 , u 2 might click d 1 under her preference to entertainment news, while chooses d 4 due to her interest in politics. when aggregating neighborhood information along the graph, different importance of neighbors under different latent preference factors should be considered. learning representations that uncover and disentangle these latent preference factors can bring enhanced expressiveness and interpretability, which nevertheless remains largely unexplored by the existing literatures on news recommendation. in this work, to address the above issues, we model the user-news interactions as a bipartite graph and propose a novel graph neural news recommendation model with unsupervised preference disentanglement (gnud). our model is able to capture the high-order connectivities underlying the user-news interactions by propagating the user and news representations along the graph. furthermore, the learned representations are disentangled by a neighborhood routing mechanism, which dynamically identifies the latent preference factors that may have caused the click between a user and news, and accordingly assigning the news to a subspace that extracts and convolutes features specific to that factor. to force each disentangled subspace to independently reflect an isolated preference, a novel preference regularizer is also designed to maximize the mutual information measuring dependency between two random variables in information theory to strengthen the relationship between the preference factors and the disentangled embeddings. it further improves the disentangled representations of users and news. to summarize, this work makes the following three contributions: ## related work in this section, we will review the related studies in three aspects, namely news recommendation, graph neural networks and disentangled representation learning. news recommendation. personalized news recommendation is an important task in natural language processing field, which has been widely explored in recent years. learning better user and news representations is a central task for news recommendation. traditional collaborative filtering (cf) based methods @xcite often utilize historical interactions between users and news to define the objective function for model training, aiming to predict a personalized ranking over a set of candidates for each user. they usually suffer from cold-start problem since news are often substituted frequently. many works attempt to take advantage of rich content information, effectively improving the recommendation performance. for example, dssm @xcite is a content-based deep neural network to rank a set of documents given a query. some works @xcite propose to improve news representations via external knowledge, and learn representations of users from their browsed news using an attention module. @xcite applied attention mechanism at both word-and news-level to model different informativeness on news content for different users. @xcite exploited different types of news information with an attentive multi-view learning framework. @xcite considered both titles and topic categories of news, and learned both long-and shortterm user representations, while @xcite represented them by multi-head attention mechanism. however, these works seldom mine highorder structure information. graph neural networks. recently, graph neu-ral networks (gnn) @xcite @xcite have received growing attentions in graph embedding because of its powerful representation learning based on node features and graph structure. @xcite explored the gnn to capture high-order connectivity information in user-item graph by propagating embeddings on it, which achieves better performance on recommendation. however, existing news recommendation methods focus on, and rely heavily on news contents. few news recommendation models consider the user-news interaction graph structure which encodes useful highorder connectivity information. @xcite modeled the user-news interactions as a graph and proposed a graph convolution based model combining long-term and short-term interests, which demonstrates the effectiveness of exploiting the user-news interaction graph structure. different from all these methods, in this work, we consider both the high-order connectivity information and latent preference factor underlying the user-news interactions. we propose a novel graph neural news recommendation model with unsupervised preference disentanglement. ## problem formulation the news recommendation problem can be formalized as follows. given the user-news historical interactions {(u, d)}, we aim to predict whether a user u i will click a candidate news d j that she has not seen before. in this paper, for a news article d, we consider the title t and profile p (a given set of entities e and their corresponding entity types c from the news content) as features. the entities e and their corresponding entity types c are already given in the datasets. each news title t consists of a word sequence @xmath0, w 2 , • • • , w m }. each profile p contains a sequence of entities defined as @xmath1, e 2 , • • • , e p } and corresponding entity types @xmath2, c 2 , • • • , c p }. we denote the title embedding as ## our proposed method in this section, we first introduce the news content information extractor which learns a news representation h d from news content. then we detail our proposed graph neural model gnud with unsupervised preference disentanglement for news recommendation. our model not only exploits the high-order structure information underlying the user-news interaction graph but also considers the different latent preference factors causing the clicks between users and news. a novel preference regularizer is also introduced to force each disentangled subspace independently reflect an isolated preference factor. ## conclusion in this paper, we consider the high-order connectivity as well as the latent preference factors underlying the user-news interactions, and propose a novel graph neural news recommendation model gnud with unsupervised preference disentanglement. our model regards the user-news interactions as a bipartite graph and encode high-order relationships among users and news by graph convolution. furthermore, the learned representations are disentangled with different latent preference factors by a neighborhood routing mechanism, enhancing expressiveness and interpretability. a preference regularizer is also designed to force each disentangled subspace to independently reflect an isolated preference, further improving the quality of user and news embeddings. experimental results on realworld news datasets demonstrate that our model achieves significant performance gains compared to state-of-the-art methods, supporting the importance of exploiting the high-order connectivity and disentangling the latent preference factors in user and news representations.