Neural News Recommendation with Collaborative News Encoding and Structural User Encoding (2021)

Abstract: Automatic news recommendation has gained much attention from the academic community and industry. Recent studies reveal that the key to this task lies within the effective representation learning of both news and users. Existing works typically encode news title and content separately while neglecting their semantic interaction, which is inadequate for news text comprehension. Besides, previous models encode user browsing history without leveraging the structural correlation of user browsed news to reflect user interests explicitly. In this work, we propose a news recommendation framework consisting of collaborative news encoding (CNE) and structural user encoding (SUE) to enhance news and user representation learning. CNE equipped with bidirectional LSTMs encodes news title and content collaboratively with cross-selection and cross-attention modules to learn semantic-interactive news representations. SUE utilizes graph convolutional networks to extract cluster-structural features of user history, followed by intra-cluster and inter-cluster attention modules to learn hierarchical user interest representations. Experiment results on the MIND dataset validate the effectiveness of our model to improve the performance of news recommendation.

## introduction online news applications, such as cnn news and msn news, have become more and more people's first choices to obtain the latest news @xcite . with a deluge of news generated every day, an efficient news recommendation system should push relevant news to users to satisfy their diverse personalized interests @xcite . from the perspective of representation learning @xcite , existing works mainly study how to effectively encode news and users into discriminative representations @xcite ; wu 1 our code is released at https://github.com/veasonsilverbullet/nnr news category title n1 travel the spookiest place in every state n2 travel the best islands in the world: 2019 readers' choice awards n3 sports long island high school football coach suspended for running up the score against previously underfeated foe n4 travel 50 states, 50 places: the top natural wonder in your state n5 sports potential florida state football coaching targets n6 sports curse of the no. 3 seed in the initial cfp rankings: how would that work? curse of the no. 3 seed in the initial cfp rankings: how would that work? the team ranked third in the initial college football playoff rankings has never made the playoffs. does that mean alabama is toast this year? we take a look at the trend and ask how would that work? distillation interpretation (b) semantic interaction between news title and content et al., 2019a; @xcite . news encoders typically extract semantic representations of news from the textual spans (e.g., news title and content). user encoders are employed to learn the representation of a user from her browsing history. news recommendation models predict the matching probabilities between candidate news and users by measuring the similarity of their representations. existing news recommendation models typically encode news title and content separately and encode users' browsing histories without explicit structural modeling. we argue that these encodings restrict the power of the news and user representations. to enhance news and user encoding, this work is established based on the two aspects of news and user representation learning: to address the above issues, in this work, we propose collaborative news encoding (cne) and structural user encoding (sue) to learn semanticinteractive news representations and hierarchical user representations. we conduct experiments on the mind dataset @xcite , showing the encoding effectiveness of our proposed model. experiments and further analyses validate that (i) cne can enhance news encoding by exploiting the word-level semantic interaction between news title and content with cross-selective and cross-attentive mechanisms; (ii) sue utilizes hierarchical cluster graphs to model the correlation of a user's browsed news, which can extract more precise user interest representations; (iii) our model significantly outperforms existing state-of-the-art news recommendation models on the real-world mind dataset. ## related work news recommendation is not only an important research task in nlp @xcite but also a core component of industrial personalized news service @xcite . conventional collaborative filtering (cf) approaches @xcite exploit the interaction relationship between news and users. since news only lasts for a short period, cf-based methods suffer from severe coldstart problem. to tackle this, content-based methods used handcrafted features to encode news and users @xcite @xcite . in recent years, deep neural models have achieved superior performance in news recommendation. many studies pinpointed that this improvement came from the fine-grained news and user representations, which were extracted by deep neural networks @xcite . for news representation learning, existing works used convolutional neural networks (cnn) @xcite , knowledge-aware cnn @xcite , personalized attention networks @xcite , and multi-head self-attention networks @xcite to extract features from news title text as news representations. @xcite employed parallel cnns to encode news title and content respectively and then concatenated them into a unified representation. @xcite encoded news title and content separately and incorporated them with multi-view attention. for user representation learning, @xcite used gru to encode user history by ordered timestamp. @xcite utilized rnn to learn short-term user representations from the browsing history, combined with long-term user embeddings. various attention networks are also widely used to attend to important news in user history @xcite . @xcite employed multi-head self-attention @xcite to capture deep interaction of user browsed news. these works formulated user history as an ordered linear sequence of news, to which recurrent or attention models were applied without modeling the structural correlation of user browsed news. hu et al. ( 2020 ) formulated news and user jointly with a bipartite graph and disentangled user preferences with routing mechanism, which however implicitly relied on the manually-set latent preference factor. ## methodology our model is composed of the collaborative news encoding (cne) module presented in section 3.1 and structural user encoding (sue) module presented in section 3.2. cne and sue extract representations of candidate news and users respectively. the overall model architecture is illustrated in figure 2. finally, section 3.3 will describe the click predictor and details of model training. ## conclusion in this work, we present a neural news recommendation model with collaborative news encoding and structural user encoding. cne leverages the title-content semantic interaction to enhance news encoding. sue exploits the correlation of browsed news and represents user interests with hierarchical cluster graphs to enhance user encoding. experiment results show that our model achieves significant performance enhancement compared to the existing state-of-the-art methods. we also further analyze our model and validate its effectiveness.