## 3 Methods of DDM
### Historically Accessed News Module  
In this module, the model mainly deals with a user's historically accessed news. The module consists of two layers. We believe that a user's historically accessed news represents the user's interests over a period of time. Therefore, processing a user's historically accessed news will result in a mathematical representation of the user's real-life interests. User interests do not tend to change in the short term. Therefore, there is inevitably a correlation between the news that users have browsed and the news they want to browse. This correlation will inevitably lead to interactions between keywords in different news articles. In this article, these correlations and interactions are used to make predictions and recommendations.  
In the first layer, this article once again uses DDM to enhance the accuracy of news recommendations by determining the interactions between news articles. The specific formula in this part is embodied in the multi-head self-attention mechanism:  
\[Attention(Q,K,V)=softmax(Qk^{T}/\sqrt{dk})V \tag{8}\]  
where Q, K, and V represent the query, key, and value matrices, respectively. They are projected onto the \(E_{t}\) matrix via different learned projection matrices. \(1/\sqrt{dk}\) is a scaling factor that generates a softer attention distribution to avoid vanishing gradients. Multi-head self-attention is applied in parallel using h-attention functions to generate connected output representations:  
\[h_{i}^{n}=Attention(E_{t}W_{i}^{Q},E_{t}W_{i}^{K},E_{t}W_{i}^{V}) \tag{9}\]  
\[h_{i}^{n}=[h_{i,1}^{n},h_{i,2}^{n},\cdots,h_{i,h}^{n}] \tag{10}\]  
where \(E_{t}\) represents the token matrix of words and \(W_{i}^{Q}\), \(W_{i}^{K}\), and \(W_{i}^{V}\) are learnable parameters, where i represents the \(i\)-th head. In the case of multi-head attention, the information of different representation subspaces at different positions can be jointly learned, which helps to capture matching signals between different words. \(h_{i}^{n}\) represents the result of feature extraction for the \(i\)-th news item, which is a combination of the results from multiple individual self-attention heads. These formulas demonstrate the functionality of the DDM module.  
\[\left\{\begin{array}{l}H=h_{i}^{n}\\ H_{1}=LN(MA(H)+H)\\ H_{2}=LN(MA(H_{1})+H_{1})\\ H_{3}=LN(MA(H_{2})+H)\end{array}\right. \tag{11}\]  
LN represents the layer normalization layer, and \(MA(H)\) represents the multi-head attention layer that comprehensively processes the output of the upper layer data. It has been observed that the best experimental results are achieved after multiple iterations of these layers. Specific demonstrations will be provided in the experimental section. Moreover, the data structure of \(h_{i}^{n}\) is the same as that of \(H_{1}\), \(H_{2}\), and \(H_{3}\). Therefore, H can be considered as a reinforced version of \(h_{i}^{n}\). In the subsequent formulas, \(h_{i}^{n}\) is used to represent the reinforced results of the H layer, i.e., \(h_{i}^{n}=H_{3}\).  
The second layer is an additive attention network. Different news articles may carry varying amounts of information that represent a user. An additive attention mechanism is used here to select important news articles to learn a more informative user representation. In the experimental section, this article also compares the additive attention mechanism to other methods such as the proximity attention mechanism. The attention weight calculation for the \(i\)-th news article is as follows:  
\[a_{i}^{n}=q_{n}^{T}\text{tanh}(V_{n}\times h_{i}^{n}+v_{n}) \tag{12}\]  
\[a_{i}^{n}=\frac{exp(a_{i}^{n})}{\sum_{j=1}^{N}exp(a_{j}^{n})} \tag{13}\]  
where \(V_{w}\) and \(v_{w}\) are projection parameters, and \(q_{n}^{T}\) is a parameter in the attention network. N represents the number of news articles being browsed. The final representation of the user's historically accessed news is obtained by weighted summing of the contextual word representations, expressed as follows:  
\[u=\sum_{i=1}^{N}a_{i}^{n}h_{i}^{n} \tag{14}\]