# A Spam Filtering Method Based on Multi-Modal Fusion
## 2 MMA-MF Model Architectures
### Fusion Model  
The structure of the fusion model is shown in Figure 3. The aim is to fuse the classification probability value of an email text part with the classification probability value of the same email image  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer (Type)** & **Output Shape** & **Param** \\ \hline conv2d\_1 (Conv2D) & (None, 128, 128, 32) & 2432 \\ batch\_normalization\_1 & (Batch (None, 128, 128, 32) & 128 \\ activation\_1 (Activation) & (None, 128, 128, 32) & 0 \\ max\_pooling2d\_1 & (MaxPooling2 (None, 64, 64, 32) & 0 \\ conv2d\_2 (Conv2D) & (None, 64, 64, 64) & 51264 \\ batch\_normalization\_2 & (Batch (None, 64, 64, 64) & 256 \\ activation\_2 (Activation) & (None, 64, 64, 64) & 0 \\ max\_pooling2d\_2 & (MaxPooling2 (None, 32, 32, 64) & 0 \\ conv2d\_3 (Conv2D) & (None, 32, 32, 64) & 102464 \\ batch\_normalization\_3 & (Batch (None, 32, 32, 64) & 256 \\ activation\_3 (Activation) & (None, 32, 32, 64) & 0 \\ max\_pooling2d\_3 & (MaxPooling2 (None, 16, 16, 64) & 0 \\ flatten\_1 (Flatten) & (None, 16384) & 0 \\ dense\_1 (Dense) & (None, 64) & 1048640 \\ batch\_normalization\_4 & (Batch (None, 64) & 256 \\ activation\_4 (Activation) & (None, 64) & 0 \\ dense\_2 (Dense) & (None, 32) & 2080 \\ dense\_3 (Dense) & (None, 2) & 66 \\ \hline \hline \end{tabular}
\end{table}
Table 3: CNN model architecture description.  
part to obtain the most accurate classification probability value of the email as spam. The overall steps are as follows: 1. Combining the two classification probability values of the LSTM and CNN models to get a feature vector \(q\), \(q\)\(\in\)\(R^{1\times 4}\); 2. Inputting \(q\) into the FC layer with 64 neurons to get a comprehensive feature vector; 3. Inputting the comprehensive feature vector to the logistic layer, which includes two neurons and chooses the logistic regression function as the activation function to get the most accurate classification probability value of the email as spam. Taking into account the efficiency of our machine, we only use the grid search optimization algorithm to select the optimal values for the four hyperparameters, which are learning rate, batch size, epochs and optimization algorithm, the best hyperparameter for learning rate is equal to 0.01, batch size is equal to 16, epochs is equal to 30 and the optimization algorithm is the SGD algorithm.  
Suppose that the classification probability dataset input to the fusion model is \(D\)={(\(q_{1}\),\(y_{1}\)), (\(q_{2}\),\(y_{2}\)), \(\cdots\), (\(q_{v}\),\(y_{v}\)), \(q_{i}\)\(\in\)\(R^{1\times 4}\), \(y_{i}\)\(\in\)[0,1], in which the conditional probability distribution of the logistic regression function is as follows:  
\[P(Y=1|q)=\pi(q)=\frac{e^{-w^{T}\cdot q}}{1+e^{-w^{T}\cdot q}}, \tag{4}\]  
\[P(Y=0|q)=1-\pi(q)=\frac{1}{1+e^{-w^{T}\cdot q}}. \tag{5}\]  
We choose the log-likelihood function as the loss function, and the formula is as follows:  
\[\begin{split} L(w)&=\sum_{i=1}^{v}[y_{i}\log\pi(q_{ i})+(1-y_{i})\log(1-\pi(q_{i}))]\\ &=\sum_{i=1}^{v}[y_{i}\log\frac{\pi(q_{i})}{1-\pi(q_{i})}+\log(1 -\pi(q_{i}))]\\ &=\sum_{i=1}^{v}[y_{i}(w\cdot q_{i})-\log(1+e^{(w\cdot q_{i})})].\end{split} \tag{6}\]  
The maximum value of \(L(w)\) is obtained by the Adam algorithm. In addition, the optimal estimate value of the parameter \(w\) can be obtained by optimizing \(L(w)\). If \(p>0.5\), it means that the email is spam; otherwise, it is a normal email.  
Figure 3: Fusion model structure.