TADI : Topic-aware Attention and Powerful Dual-encoder Interaction for Recall in News Recommendation (2023)

Abstract: News recommendation is one of the widest commercialization in natural language processing research area, which aims to recommend news according to user interests. New recall plays an important role in news recommendation. It is to recall candidates from a very large news database. Recent researches of news recall mostly adopt dual-encoder architecture as it provides a much faster recall scheme, and they encode each word equally. However, these works remain two challenges: irrelevant word distraction and weak dual-encoder interaction. Therefore, we propose a model Topic-aware Attention and powerful Dual-encoder Interaction for Recall in news recommendation (TADI). To avoid irrelevant word distraction, TADI designs a Topic-aware Attention (TA) which weights words according to news topics. To enhance dual-encoder interaction, TADI provides a cheap yet powerful interaction module, namely Dual-encoder Interaction (DI). DI helps dual encoders interact powerfully based on two aux targets. After performance comparisons between TADI and state-of-the-arts in a series of experiments, we verify the effectiveness of TADI.

## introduction news recommendation is one of the widest commercialization in natural language processing research area, which feeds rich and suitable news to users based on their interests. currently, news recommendation is generally used on online news websites, movie review websites and etc (such as msn news), it thus has become an useful tools to provide masses of custom information in one go. generally, recall and ranking are two main steps of news recommendation @xcite . the first one is to recall candidates from a very large news database, while the second one is to rank news candidates for display. news recall determines the room of recommendation performance, and thus this paper discusses about it. researches of news recall @xcite (or new candidate generation @xcite or news retrieve @xcite ) can be categorized into feature-based and content-based models. feature-based models focus on feature interaction modeling such as youtubenet @xcite and pinnersage @xcite . feature-based models require to summarize features by manual text mining and thus they inevitably lose useful information. with the development of content understanding technology, researchers incorporate content understanding with feature interaction, that is content-based models, such as @xcite . different from manual text mining of feature-based models, contentbased models directly learn representations of user and news by modeling news contents. however, content-based models ignore irrelevant word distraction problem. every word is encoded equally, that is why irrelevant words would bring sideeffect for news recommendation. for example, football fans are more interested in the news "lionel messi comes back spain for taking a holiday" than tourists, but only two words "lionel messi" in the title are related. figure 1 : the typical dual-encoder model architecture @xcite . to recall news candidates, researches mostly rely on dual-encoder architecture which is shown in figure 1 . dual-encoder architecture is able to serve efficiently in real time for a large scale of news. because it encodes user and news inde-pendently, and solves the top-k nearest neighbor search problem in sublinear complexity, by converting the problem to maximum-inner-product search (mips) @xcite . however, there exists a classical challenge of dual-encoder architecture @xcite , that is weak dual-encoder interaction. specifically, the click predictor (e.g., dot product), which unfortunately is the only interation between dual encoders in that architecture. weak interaction makes one encoder difficultly utilize the information of the other encoder. therefore, model underestimates actual correlation between dual encoders, resulting in severe performance degradation. to response to the aforementioned challenges, we propose a news recall model, namely, topicaware attention and powerful dual-encoder interaction for recall in news recommendation (tadi) foot_0 . first, we design the topic-aware attention (ta) mechanism to avoid irrelevant word distraction. because news topic is one of the most important interest indicators which directly reflects preference of potential target users. so it is reasonable to weight words by using ta. secondly, tadi involves the dual-encoder interaction (di) module which helps dual encoders interact more powerful than typical dualencoder models. in detail, di provides two auxiliary targets to enhance dual encoders interaction on training, without changing the mechanism of online news recall in dual-encoder architecture. therefore, tadi can leverage the efficiency of dual-encoder architecture on news recall while simultaneously gaining the ability to more powerful interaction. afterwards, the effectiveness of tadi is verified by conducting a series of experiments on benchmark dataset mind @xcite . in summary, our contributions are four-fold: (1) we propose a news recall model tadi. (2) we design the topic-aware attention ta to avoid the distraction of irrelevant words. (3) we propose the dual-encoder interaction di to enhance dualencoder interaction. (4) extensive experiments are conducted on the benchmark datasets, which demonstrate the effectiveness of tadi. ## related works researches of news recall have rather matured works, mainly could be divided into fearture-based models and content-based models. furthermore, we introduce researches of news ranking because its technologies are available for news recall. ## problem formulation in this section, we give main notations and define the news recall problem. features of news and user are the same as previous works @xcite . first, for news candidate defination, we make it represented by four type features: title, category, sub-category and title entity. the news title t n is a word sequence. denote the category and sub-category of the news are c n and s n . denote the title entity as d n where consists of entities. secondly, we assume a user has n historical clicked news, and structure of each historical news representation is the same as news candidate. denote titles as @xmath0, t u 2 , ..., t u n ], categories as @xmath1, c u 2 , ..., c u n ], sub-categories as @xmath2, s u 2 , ..., s u n ] and title entities as ## the proposed model tadi is divided into four modules, i.e., user encoder, news encoder, predictor, and dual-encoder interaction, which is shown in figure 2 . the user encoder and the news encoder respectively generate embeddings of user and news. the predictor calculates dot product between embeddings of user and news, in order to predict click probability. the dual-encoder interaction provides a capability which helps dual encoders interact more powerful. ## conclusion in this paper, we propose the model tadi for news recommendation. to avoid irrelevant word distraction, tadi designs the topic-aware attention (ta). ta uses news topic to weight words since news topic is one of important insterest indicators which directly reflects preference of potential target users. to make dual encoders interaction more powerful, tadi provides dual-encoder interaction (di). di helps dual-encoder interact more powerfully by providing two auxiliary targets. after conducting a series of experiments, the effectiveness of the proposed tadi is verified. besides, extensive analyses demonstrate the robustness of the proposed model. in the future, we plan to achieve better performance by tadi optimization and data augmentation usage.