# Stacking-Based Ensemble Learning of Self-Media Data for Marketing Intention Detection
## 4 Experiment Settings
### Metrics  
In the experiment, F1, Accuracy (Acc), and the Logarithmic Loss Function (Loss) were used as L = logarithmic Loss indicators. F1 is the weighted harmonic average of Precision (P) and Recall (R). F1 is an appropriate method to measure the reliability of the model. Accuracy represents the ratio of the number of classified correctly to the total number of the dataset. Loss is used to evaluate the inconsistency between the predicted result and the real result of the model. It is a function of non-negative real value. The smaller the Loss function is, the better the robustness of the model will be. The calculation formulas are shown as follows., where Y indicates the variable of the output variable, X the input variable, L the Loss function, N the sample size of the input, while \(y_{i}\) is the real category of input instance, and \(p_{i}\) the probability that the input instance belongs to Category 1. \[R=\frac{TP}{TP+FN} \tag{9}\]  
\[P=\frac{TP}{TP+FP} \tag{10}\]  
\[F1=\frac{2 PR}{P+R} \tag{11}\]  
\[ACC=\frac{(TP+TN)}{P+N} \tag{12}\]  
\[L(Y,P(Y|X))=-\frac{1}{N}\sum_{i=1}^{N}\left(y_{i}\log p_{i}+(1-y_{i})\log\left( 1-p_{i}\right)\right) \tag{13}\] \begin{table}
\begin{tabular}{c c c c c c c} \hline
**Corpus** & **Num\_Topics** & **id2 Word** & **Chunksize** & **Decay** & **Size** & **Sentence** & **min\_Count** \\ \hline corpus\_tfidf & 2 & dictionary & 2000 & 1 & 50 & corpus & 5 \\ \hline \end{tabular}
\end{table} 
Table 1: The model parameters of LSI-Word2 vec. \begin{table}
\begin{tabular}{c c c c c} \hline
**Model** & **max\_Depth** & **n\_Estimators** & **Num\_Leaves** & **cv** & **min\_Samples** \\ \hline Gradient Boosting & 13 & 100 & - & - & - \\ Random Forest & - & 32 & - & - & 2 \\ XGBoost & 3 & - & - & - & - \\ LightGBM & - & 2000 & 32 & - & - \\ Stacking & - & - & - & 5 & - \\ \hline \end{tabular}
\end{table} 
Table 2: The model parameters of Stacking. Table 3 shows the nature of three various feature extractions on account of the same dataset and the same classifier. By comparing the indicators of Acc and F1, it can be seen that LSI-Word2 vec can fully extract semantic information to achieve a better classification effect. In order to find the most suitable dimension of the method of feature extraction, the methods were compared according to different word vector dimensions under different feature extraction methods. The following is a comparison of the experimental results based on the stacking model in different dimensions, using different feature extraction methods. From Figure 6, it can be seen that the word vector dimension cannot be too low, and it is more appropriate to choose 100 or so. The following is an experiment based on the feature extraction method of LSI-Word2 Vec. The news recognition model included the results of the first layer of various decision trees of the stacking model and the results of the second layer integrating the models. The pruning-stacking model represents the result of post-pruning integration of decision trees. The experimental results are as follows in Table 4. The stacking integration strategy can greatly enhance the performance of the classifier after multiple trainings. By comparing the running time of the unpruned decision tree model in the first layer of the stacking model to the running time of the decision tree after pruning, it can be seen that the computational time of the pruning strategy was greatly reduced. Compared with other classifier models, the model constructed in this paper can achieve better results. This model performed well in marketing intent recognition and can be executed more efficiently. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Model** & **F1** & **Acc** & **Loss** & **Time** \\ \hline Gradient Boosting & 0.733 & 0.846 & 6.315 & - \\ Random Forest & 0.720 & 0.831 & 6.828 & - \\ XGBoost & 0.701 & 0.847 & 6.225 & - \\ LightGBM & 0.737 & 0.846 & 6.312 & 280 \\ Stacking & 0.739 & 0.848 & 6.222 & 220 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of the classification results.