## 4 Methodology
### Multi-Modal Attention Fusion  
In order to incorporate the multi-modal information, we introduce an attention mechanism to focus on specific features, since each feature contributes differently to the model. The architecture is show in Figure 13. This is followed by multiple layers of the perceptron. Taking the concatenation of the representation encoded by different modules denoted as \(\mathbf{Z}\in\mathbb{R}^{1\times D}\) as input, it outputs one final unit \(y\), i.e., a cascade size level (\(y=log(Y^{s}_{t_{p}=\infty}+1)\)). \(D\) represents the sum of the dimensions of the four output embeddings of the uni-modal representation extraction, i.e., \(D=d_{n}+d_{tw}+d_{u}+d_{tr}\).  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Feature** & **Description** \\ \hline Statistical & _img\_count_ & Number of images in the news article. \\ \hline  & \(RSI\_Mean\) & \begin{tabular}{c} _Relative strength index_: A momentum indicator that measures \\ the magnitude of recent trend changes. The average is the three \\ keywords’ RSI values. \\ \end{tabular} \\  & \(RSI\_Max\) & \
\begin{tabular}{c} There are three variables for different time-scales. \\ We use the largest RSI value of the three keywords. \\ There are three variables for different time-scales. \\ \end{tabular} \\ Trend &
\begin{tabular}{c} _MA_: moving average is a statistic that captures the average \\ change in a data series over time. The average is the three \\ keywords’ \(MA\) values. \\ There are three variables for different time-scales. \\ We use the largest \(MA\) value of the three keywords. \\ There are three variables for different time-scales. \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table}
Table 6: Trend features.  
The neural network architecture that implements the attention mechanism can be formulated as  
\[y=\sigma(\mathbf{W}_{\ell_{2}}(\sigma(\mathbf{W}_{\ell_{1}}(\mathbf{Z}\odot\alpha)+\mathbf{b}_{ \ell_{1}}))+\mathbf{b}_{\ell_{2}}), \tag{9}\]  
in which \(\sigma\) corresponds to the activation function ReLU, \(\odot\) refer to the element-wise product, and \(\mathbf{W}_{\ell_{1}},\mathbf{W}_{\ell_{2}},\mathbf{b}_{\ell_{1}},\mathbf{b}_{\ell_{2}}\) are learnable weights.  
\[\alpha=softmax(W_{l_{attn}}\mathbf{Z}+b_{l_{attn}}), \tag{10}\]  
where \(\mathbf{W}_{l_{attn}}\in\mathbb{R}^{D\times D}\); \(\mathbf{b}_{l_{attn}}\in\mathbb{R}^{1\times D}\); \(\alpha\in\mathbb{R}^{1\times D}\) denotes the attention score vector, with the value of each dimension in the vector \(\alpha_{i}\in\mathbb{R},1<i<D\) representing the importance of the feature corresponding to this dimension. To obtain our target, the cascade size level \(\hat{y}\), we applied a multilayer perceptron (MLP) with one hidden layer on \(\mathbf{Z}\odot\alpha\in\mathbb{R}^{1\times D}\) to obtain the target.  
**Objection function.** The objective function to minimize is defined as follows:  
\[L(\hat{\mathbf{Y}},\mathbf{Y})=\frac{1}{S}\sum_{i}^{S}(log(\hat{y}_{i}+1)-log(y_{i}+1) )^{2}, \tag{11}\]  
in which \(S\) is the total number of cascades, \(\hat{\mathbf{Y}}\) is the predicted value, and \(\mathbf{Y}\) is the ground-truth label. The cascade size, or popularity of a tweet, is measured on a natural logarithmic scale. This is because tweet popularity can vary greatly, ranging from just one retweet to thousands. Most tweets have very few retweets, resulting in a heavy and long-tailed distribution, as illustrated in Figure 6. When using logarithmic scaling, the model is not impacted by extreme values, which ensures stability for the loss function.