# Feature-Interaction-Enhanced Sequential Transformer for Click-Through Rate Prediction
## 4 Experiments
### Experiment Settings
#### 4.1.2 Baseline Models  
To confirm the superiority of the proposed model, we compare it with eleven representative feature interaction baselines in the past seven years, six representative sequential recommendation baselines in the past five years, and one representative baseline that combines feature interactions and sequential recommendations. The following models are representative feature interaction baselines:  
* **Wide and Deep**[5]: uses a parallelization of a linear model and a DNN model to extract both the low-order and the high-order feature combinations.
* **IPNN**[4]: learns two-order feature interactions based on inner products and high-order feature interactions by stacking multiple layers of the DNN network.
* **DeepFM**[6]: an end-to-end model similar to Wide and Deep except that it replaces the linear model with an FM model and shares the same input for both the wide and the deep part.
* **DCN**[7]: consists of cross network layers and a DNN network in parallel.
* **AutoInt**[11]: it stacks multi-head self-attention layers to extract high-order feature interactions.
* **ONN**[43]: proposes operation-aware embedding layers to learn diverse representations of each feature field for different operations.
* **DCNv2**[29]: improves the cross network in DCN, enabling both point-wise and vector-wise feature interactions.
* **DESTINE**[30]: adopts disentangled self-attention layers to extract high-order feature interactions.
* **DualMLP**[8]: a well-tuned two-stream MLP model.
* **FinalMLP**[8]: an enhanced two-stream MLP model combined with stream-specific feature gating and bilinear interaction fusion layers.
* **FINAL**[31]: Takes advantage of multiplicative feature gating for soft feature selection and multiplicative feature interactions to achieve exponential degree growth. Self knowledge distillation is used to supervise the outputs of two FINAL blocks.  
The following models are representative sequential baselines:  
* **DIN**[13]: introduces the attention mechanism by adaptively attending to historical user behaviors to learn the user's interest.  
\begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Dataset** & **\# Users** & **\# Items** & **\# Samples** & \
\begin{tabular}{c} **\# Sequence** \\ **Fields** \\ \end{tabular} &
\begin{tabular}{c} **\# Non-Sequence** \\ **Fields** \\ \end{tabular} \\ \hline Ele.me & 1,013,338 & 449,440 & 1,228,589 & 14 & 9 \\ Bundle & 118,931 & 11 & 4,193,418 & 24 & 9 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistics of the datasets.  
* **SASRec**[21]: Utilizes the causal Transformer model for CTR prediction. The latent factor model in the prediction layer is replaced with an MLP model for binary classification.
* **DIEN**[44]: based on AUGRU to capture the evolution of the user interest relevant to the target item.
* **BST**[22]: utilizes the Transformer encoder layer to capture the sequence information of the user's behaviors.
* **TiSASRec**[41]: Embeds both the absolute positions and the relative time intervals of the user interaction sequences and designs a time-interval-aware self-attention mechanism to capture the temporal dependencies of the user action. The latent factor model in the prediction layer is replaced with an MLP model for binary classification.
* **DMR**[24]: An item-to-item network is to learn the user temporal interest representation and the implicit user-item relevance. A user-to-item network is to learn the explicit user-item similarity by using the inner product.  
The following model is a representative baseline that unifies feature interaction and sequential recommendation:  
**JointCTR**[23]: A joint framework that combines a logistic regression model, an FM model, a multi-head attention network, and a sequential feature learning model based on DIEN in a parallel manner.