# News Recommendation Method Based on Candidate-Aware Long- and Short-Term Preference Modeling
## 3 Method
### News Encoder  
There are usually multiple components in news, so modeling multiple components of the news collaboratively helps to obtain high-quality news representations. The news encoder designed in this paper learns news representations from four different perspectives: title, abstract, topic, and entities included in the title and abstract of the news. Furthermore, since news titles and abstracts are often potentially linked, this paper captures this potential link by using multi-head cross-attention networks, which is a variant of multi-head self-attention. As shown in Figure 4, the news encoder contains the following main modules:  
**Word embedding layer. ** Since words cannot be used directly as input to the model, a medium is needed to map words to numbers that the model can understand. The word embedding layer is exactly this medium, which maps words into multidimensional vector representations. The word embedding layer maps words into high-dimensional vector representations. After the word embedding module, the news title, abstract, and category are mapped, respectively, as \(\left[\mathbf{e}(\mathbf{w}_{1}^{t}),\mathbf{e}(\mathbf{w}_{2}^{t}),\ldots, \mathbf{e}(\mathbf{w}_{N}^{t})\right]^{\mathbf{T}}\), \(\left[\mathbf{e}(\mathbf{w}_{1}^{b}),\mathbf{e}(\mathbf{w}_{2}^{b}),\ldots, \mathbf{e}(\mathbf{w}_{N}^{b})\right]^{\mathbf{T}}\) and  
Figure 3: The architecture of our model. \([\mathbf{e}(\mathbf{c})]^{T}\). \(\mathbf{e}(\cdot)\) denotes the word embedding. Since the topic embedding is only involved in the final feature aggregation, it is denoted as \(\mathbf{r}^{\mathbf{c}}\) in this paper for convenience. **Attention layer. ** Additive attention networks are widely used in news semantic information extraction. For news titles or abstracts, different words have different informativeness. Therefore, additive attention networks were applied to filter out the important words in them to better represent news titles. Inspired by previous works [15], the weight of the \(i\)-th word \(a_{i}^{t}\) is denoted as follows:  
\[\mathbf{a_{i}^{t}}=\mathbf{q_{i}^{T}}\tanh\Bigl{(}\mathbf{V_{t}}\times \mathbf{e}(\mathbf{w}_{t}^{t})+\mathbf{b_{t}}\Bigr{)} \tag{1}\]  
\[\alpha_{i}^{t}=\frac{\exp\left(\mathbf{a_{i}^{t}}\right)}{\sum\limits_{j=1}^{ t}\exp\left(\mathbf{a_{j}^{t}}\right)} \tag{2}\]  
where \(\mathbf{q_{t}}\) is an attention query vector, \(\mathbf{V_{t}}\) and \(\mathbf{b_{t}}\) are trainable parameter matrices. By calculating the sum of the products of the vector representations of the words and their attention weights, we obtain a representation of the news title,  
\[\mathbf{n^{t}}=\sum\limits_{j=1}^{t}\alpha_{j}^{t}\mathbf{e}\Bigl{(}\mathbf{ w}_{j}^{t}\Bigr{)} \tag{3}\]  
Similarly, the representation of the news abstract \(\mathbf{n^{b}}\) can be obtained. **Multi-head cross-attention networks. ** News titles and abstracts usually contain some obscure words. For example, in the title of the example article, \"[PERSON] admits talks with Washington, but it \"takes two to tango\", the meaning of \"tango\" is not as literal as it seems. The second half of the abstract, \"he has talked to Washington president [PERSON]\", makes a good complement to the meaning of \"tango\". Considering news abstracts as external knowledge enriches the semantics of the titles. Taking title representation \(\mathbf{n^{t}}\) and abstract representation \(\mathbf{n^{b}}\) as inputs, the title representation is used as a query vector to mine potential links between title and abstracts. After multi-head cross-attention networks, the news abstract representation \(\mathbf{r^{b}}\) is computed as follows:  
\[\mathbf{r^{b}}=\text{MultiHeadCrossAttention}\Bigl{(}Q=\mathbf{W_{t}} \mathbf{n^{t}},K=\mathbf{n^{b}},V=\mathbf{n^{b}}\Bigr{)} \tag{4}\]  
Similarly, we use the abstract representation \(\mathbf{n^{b}}\) as a query vector to mine potential links between the abstract and title.