# Information Fusion-Based Deep Neural Attentive Matrix Factorization Recommendation
## 3 Preliminary Work
### Cross-Features Fusion Based on Attention Mechanism  
After obtaining the cross-features of the user and the item through the interaction layer, we need to fuse these cross-features obtained by feature crossing of different information to finally obtain the user preference representation and the item representation, namely the potential feature vector of the user and the item, \(p_{u}\) and \(q_{i}\). However, it should be noticed that different cross-features have different degrees of importance on the final prediction result. Some cross-features may directly affect the final recommendation result. On the contrary, some cross-features are not important for the final result. If we combine these cross-features with the same weight without distinguishing the importance of these cross-features, these insignificant cross-features may become noise and offset the positive impact of the important cross-features on the final recommendation result, which ultimately affects the performance of recommendation model. Here, we still take the Disney advertising click prediction scenario as an example. The cross-feature obtained by the cross of age and date has a great influence on the click prediction result, and the cross-feature obtained by the cross of height and accent is irrelevant for users' clicks on Disney advertising. For two users, assuming that the cross-features representation obtained by the cross of their age and date are very different, the final click prediction result should be greatly different. However, if the two cross-features are fused with a weight of 1, the possible result is that the cross-feature obtained by the cross of height and accent makes up for the great difference of the cross-feature obtained by the cross of age and date, which cause the two users' click prediction results are same. Therefore, it is necessary to distinguish the contribution of these cross-features on the final result by different importance levels. Here, we adopt the attention mechanism to learn the importance of different cross-features on prediction results, by which we fuse different cross-features. Specifically, by applying the attention net of the pooling layer based on the attention mechanism on the different cross-features, the attention scores of different cross-features of users and items \(a_{ij}\) and \(a^{{}^{\prime}}_{ij}\) are, respectively, learned, which represents the contribution of different cross-features to the prediction. The calculation formula is formulated as follows:  
\[a_{ij}(U_{t},U_{t^{\prime}})=\frac{e^{h^{T}ReLU(W(U_{t}\odot U_{t^{\prime}})+b)} }{\sum_{U_{t}\odot U_{t^{\prime}}\in\varphi_{pairwise}(u)}e^{h^{T}ReLU(W(U_{t} \odot U_{t^{\prime}})+b)}} \tag{3}\]  
\[a^{{}^{\prime}}_{ij}(I_{t},I_{t^{\prime}})=\frac{e^{h^{T}ReLU(W(U_{t}\odot I_{t ^{\prime}})+b)}}{\sum_{U_{t}\odot I_{t^{\prime}}\in\varphi_{pairwise}(i)}e^{h^{ T}ReLU(W(I_{t}\odot I_{t^{\prime}})+b)}} \tag{4}\]  
where \(W\) and \(b\) represent the weight matrix and bias vector of the interaction layer to the attention net, respectively. \(h\) represents the connection weight of attention net to the pooling layer.  
Then, multiple cross-features of the user and the item are combined according to different contribution levels through the sum pooling operation, thereby obtaining the potential feature vector containing the attribute features of the user and the item, namely \(p_{u}\) and \(q_{i}\). The calculation process is formulated as follows:  
\[p_{u}=\varphi(\varphi_{pairwise}(u,a_{ij}(U_{t},U_{t^{\prime}})))=\sum_{t\
eq t ^{\prime}}a_{ij}(U_{t},U_{t^{\prime}})U_{t}\odot U_{t^{\prime}} \tag{5}\]  
\[q_{i}=\varphi(\varphi_{pairwise}(i,a^{{}^{\prime}}_{ij}(I_{t},I_{t^{\prime}}))) =\sum_{t\
eq t^{\prime}}a^{{}^{\prime}}_{ij}(I_{t},I_{t^{\prime}})I_{t} \odot I_{t^{\prime}} \tag{6}\]