SimCEN: Simple Contrast-enhanced Network for CTR Prediction

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 08:13:30

Abstract: Click-through rate (CTR) prediction is an essential component of industrial multimedia recommendation, and the key to enhancing the accuracy of CTR prediction lies in the effective modeling of feature interactions using rich user profiles, item attributes, and contextual information. Most of the current deep CTR models resort to parallel or stacked structures to break through the performance bottleneck of Multi-Layer Perceptron (MLP). However, we identify two limitations in these models: (1) parallel or stacked structures often treat explicit and implicit components as isolated entities, leading to a loss of mutual information; (2) traditional CTR models, whether in terms of supervision signals or interaction methods, lack the ability to filter out noise information, thereby limiting the effectiveness of the models.
  
  In response to this gap, this paper introduces a novel model by integrating alternate structure and contrastive learning into only one simple MLP, discarding the design of multiple MLPs responsible for different semantic spaces, named the Simple Contrast-enhanced Network (SimCEN), which employs a contrastive product to build second-order feature interactions that have the same semantic but different representation spaces. Additionally, it employs an external-gated mechanism between linear layers to facilitate explicit learning of feature interactions and to filter out noise. At the final representation layer of the MLP, a contrastive loss is incorporated to help the MLP obtain self-supervised signals for higher-quality representations. Experiments conducted on six real-world datasets demonstrate the effectiveness and compatibility of this simple framework, which can serve as a substitute for MLP to enhance various representative baselines. Our source code and detailed running logs will be made available at https://anonymous.4open.science/r/SimCEN-8E21.

Main paper: 
 revisiting explicit & implicit paradigm for ctr prediction 2.1 multimodal feature embedding: 
 In the explicit & implicit paradigm, feature embedding is a commonly used technique that maps high-dimensional and sparse raw features into dense and continuous representations: e 𝑖 = E 𝑖 𝑥 𝑖 , where E 𝑖 ∈ R 𝑑 ×𝑠 𝑖 and 𝑠 𝑖 separately indicate the embedding matrix and the vocabulary size for the 𝑖 𝑡ℎ field, 𝑑 represents the embedding dimension. Subsequently, we can obtain the result representation of the embedding layer: E = e 1 ; e 2 ; • • • ; e 𝑓 ∈ R 𝑓 ×𝑑 , where 𝑓 denotes the number of fields. S 𝑖 = segment(E), where S 𝑖 represents the 𝑖 𝑡ℎ semantic space and the segment represents various augmentation or no operations, such as gating mechanisms, product operation, adding noise and so on.For multimodal feature data such as micro-videos, their thumbnails can be preprocessed using a visual model (e.g., ResNet [16] ) to produce high-dimensional visual embeddings that are associated with corresponding category labels. These embeddings can then be further reduced in dimensionality through Principal Component Analysis (PCA) [1] , which decreases the computational cost of the model while preserving essential features. As for features containing timestamps, such as a user's sequence of click behaviors, average pooling can be employed to integrate the high-dimensional and variable behavioral sequences over time into a stable representation, enabling interaction with other low-dimensional features.Parallel structures [48, 52] typically employ two concurrent components, explicit & implicit, that attempt to complement the performance bottleneck of MLPs by leveraging low-order explicit feature interactions in different semantic spaces. Formally, the fusion scheme for the parallel structure is defined as follows:EQUATIONŷ = F 𝑓 𝑢𝑠𝑖𝑜𝑛 (V 𝑒𝑥𝑝 1 , V 𝑒𝑥𝑝 2 , . . . , V 𝑒𝑥𝑝 𝑛 , V 𝑖𝑚𝑝 1 , V 𝑖𝑚𝑝 2 , . . . , V 𝑖𝑚𝑝 𝑛 ), (2) where V 𝑒𝑥𝑝 𝑖represents the output of the explicit component in the 𝑖 𝑡ℎ semantic space, V 𝑖𝑚𝑝 𝑖 denotes the result of the implicit capture in the semantic space. F 𝑓 𝑢𝑠𝑖𝑜𝑛 is the aggregation function. ŷ represents the final prediction value of the model. The parallel structure attempts to simultaneously capture explicit and implicit feature interaction information to achieve a complementary effect.Stacked structures [38, 60] utilize explicit interactions based on product operations to augment the information within the original semantic space. Simply put, this idea seeks to directly enrich the MLP's input by introducing the product information that it typically finds difficult to learn. Formally, the definition of stacked structures is as follows:V 𝑒𝑥𝑝 = explicit(E), EQUATIONIt is clear that in both parallel and stacked structures, the explicit and implicit components exist independently as parts of the model, rather than being unified. Therefore, the alternate structure intersperses explicit interactions within the implicit ones, aiding in the better acquisition of mutual information between the explicit and implicit components. More specifically, we introduce explicit product operations at every linear layer of the MLP, allowing the explicit and implicit components to be integrated as a whole, rather than as separate entities. The definition of the alternate structure is as follows:EQUATIONwhere 𝑛 represents the number of suitable semantic spaces, and V 𝑎𝑙𝑡 𝑙 denotes the output of the 𝑙 𝑡ℎ layer of the alternate structure. This structure alternates between implicit and explicit components to model feature interactions. efficacy in predicting the true CTR. We conjecture that this might be due to the alternate structure helping the MLP learn the product operation layer by layer, which is difficult for it to capture on its own [42] , resulting in more accurate predictions. Predicting user CTR is a classic binary classification problem [39, 54] . To further investigate the discrepancies between the final predictions of the three structures and the true labels ∈ {0, 1}, we plot the CTR prediction distributions for positive and negative samples separately using a polar coordinate system (randomly sampling 1,000 instances). The corresponding predictions are recorded when each structure is performing optimally. The visualizations are shown in Figures 3 and 2 . To more intuitively represent the differences between prediction and true values, we calculate the average distance ( 1 1000 1000 𝑖=1 | ŷ𝑖 -0|) of the predicted values from the center point in each polar coordinate system. For negative samples (true label = 0), a shorter average distance is preferable, indicating more accurate model predictions (i.e., the distribution of predictions is more clustered around the center point), while for positive samples, the opposite is true.By correlating Figures 3 and 2 with the corresponding Logloss in Table 1 , we can preliminarily demonstrate the effectiveness of the alternate structure. The prediction distribution of the alternate structure for negative samples is more concentrated around the center point, with a shorter average distance. In contrast, for positive samples, the prediction values tend to be uniformly distributed toward (1, 𝜃 ), with a longer average distance. Similarly, Logloss is a metric that measures the difference between the predicted probability by the model and the actual occurrence probability [39, 50] . Therefore, when the model can more accurately classify both negative and positive samples, Logloss decreases. The alternate structure excels in both aspects, resulting in lower Logloss values as shown in Table 1 compared to other structures. 
simcen: simple contrast-enhanced network for ctr prediction: 
 Based on the results from Section 2, we have demonstrated the efficacy of the alternate structure. Thus, in this section, to further explore the potential of this structure, we attempt to enhance the advantages of this structure by utilizing concepts related to contrastive learning. Next, we will introduce the SimCEN model from the bottom up, with its architecture depicted in Figure 4 .As mentioned before, some existing studies [38, 39, 41, 42] have extensively demonstrated the effectiveness of the inner product. By combining feature pairs, not only assists MLP in learning the inner product operation but also augments the data, thereby achieving better performance. However, these studies only consider the upper triangular elements of the inner product matrix, not the full elements. Therefore, we propose the concept of the contrastive product to delineate two semantically identical but representationally distinct second-order feature interaction spaces, allowing for contrastive learning between the upper and lower triangular elements. The formulated representation of the contrastive product is as follows:EQUATIONwhere E ∈ R 𝑓 ×𝑑 denotes feature embeddings, 𝜙 ∈ R 𝑑 ×𝑑 is learnable weight matrix, Upper refers to the upper triangular part of the inner product matrix, and likewise, Under refers to the under triangular part, W 𝑢𝑝 andW 𝑢𝑛 ∈ R 𝑓 ( 𝑓 +1) 2×𝐷 are two transformation matrices.S v 1 and S v 2 represent 2-order (o 2 ) feature interactions that have the same semantic but different representation spaces. Indeed, this approach of expanding semantic spaces can yield a greater number of spaces. However, due to considerations of time complexity, we choose to expand only two additional semantic spaces.Gating mechanisms [11, 27, 34, 48, 54] are widely applied in CTR prediction, but most of models utilize a self-gated mechanism [40] ,where the gating signal is generated by the input information itself, rather than depending on an external input: F 𝐺𝑎𝑡𝑒 (S) = S⊙Gate(S). However, it is evident that such a gating mechanism can only act as an information filter and is incapable of performing crucial interaction operations in CTR prediction. Therefore, we use a new external-gated mechanism to create interaction signals between representations of different semantic spaces and filter information:EQUATIONwhere W s and b s are weight and bias. 𝛼 is a learnable parameter that can adaptively scale the range of the sigmoid function (𝜎), thereby obtaining a more dynamic interaction and gating capability.As defined in Section 2.4, when we construct an alternate structure, the order in which components are arranged inevitably arises as an issue (akin to the sequence of batch norm and linear layers). However, after our experiment, there is less difference in the performance of alternate structures in either order. For clarity of exposition, we default to describing the structure in an implicit before explicit sequence. The formalization of the alternate interaction is as follows:• Input Layer: To enhance the diversity of semantic information, we utilize the contrastive product to obtain two additional secondorder feature semantic spaces, which are then concatenated with the S ego = flatten(E):EQUATION• Interaction Layer: To further diversify the information captured by the linear layer across three semantic spaces without losing global information, we embrace a divide-and-conquer approach. When V 𝑎𝑙𝑡 𝑖 passes through a common linear layer, it is divided into three parts, each processed separately:EQUATIONwhere W V 𝑙 and b V 𝑙 are the weight and bias of the linear layer, chunk represents the split operation (i.e., the inverse operation of concatenation), ego, v 1 , and v 2 respectively represent the temporary representations of three semantic spaces (S ego , S v 1 , S v 2 ). Δ represents random perturbation sampled from a uniform distribution, and 𝜎 (•) denotes different activation functions.• Fusion Layer: To ensure the model captures both the diversity and homogeneity of information, we further introduce L 𝑐𝑙 and intra-layer connection (ILC):EQUATIONEQUATIONEQUATIONwhere the temperature coefficient 𝜏 plays a regulatory role, B represents the batch size, and 𝑖, 𝑗 represents the sample index. M represents a mask randomly sampled from a Bernoulli distribution. The intra-layer connection can be seen as a form of horizontally skip connection [16] that introduces a mask. Empirically, this approach can ensure the homogeneity of v 1 , v 2 , and ego with a probability 𝑚. • Training: We combine the widely used binary cross entropy (Logloss) [39, 50, 52, 65] with contrastive loss as our total loss function:EQUATIONwhere 𝜆 represents hyperparameters that control the balance between the loss functions, 𝑁 is the total number of training samples, and 𝑦 represents the true label.In our model, unlike the approach in InfoNCE [35] , we do not enforce consistency between the two perturbed augmented semantic spaces v 1 𝐿 and v 2 𝐿 . Instead, we encourage consistency between the ego and them. In CTR prediction, users may exhibit diverse interests (i.e., multi-interest), manifesting as vastly different click behaviors, thus blindly introducing the alignment concept of contrastive learning often deteriorates feature representation learning [15] . As mentioned in CETN [25] , by considering the diversity and homogeneity of representations from different semantic spaces, we can introduce the L 𝑐𝑙 to help the model learn richer and higher-quality feature information. Simultaneously, we introduced a product-based interaction operation between linear layers through the external-gated mechanism, thereby finer-grained decoupling the representation learning process of explicit and implicit feature interactions. This strengthens the fusion and communication of information across different semantic spaces and reduces feature interaction noise. 
experiments 4.1 experimental settings: 
 Datasets and preprocessing. We evaluate SimCEN on six realworld datasets: Avazu 1 [54], Criteo 2 [65] , MovieLens 3 [7] , Frappe 4 [2, 7], MicroVideo 5 [5] , and KuaiVideo 6 [26] . Table 2 provides detailed information about these datasets. For data preprocessing methods, we follow the settings from [65] 7 .Evaluation metrics. To compare the performance, we utilize two commonly used metrics in CTR models: Logloss and AUC [14, 38, 43, 48] . Logloss is the calculation result of binary cross entropy. A lower Logloss suggests a better capacity for fitting the true data (i.e., classification capability). AUC stands for Area Under the ROC Curve, which measures the probability that a positive instance will 1 https://www.kaggle.com/c/avazu-ctr-prediction 2 https://www.kaggle.com/c/criteo-display-ad-challenge 3 https://grouplens.org/datasets/movielens/ 4 http://baltrunas.info/research-menu/frappe 5 https://huggingface.co/datasets/reczoo/MicroVideo1.7M_x1/tree/main 6 https://huggingface.co/datasets/reczoo/KuaiVideo_x1/tree/main 7 https://github.com/reczoo/BARS/tree/main/datasets Table 3 : Performance comparison of different models. "*": Integrating the original model with DNN networks. We bold the performance of SimCEN and related models, while underlined scores are the second best. Meanwhile, we conducted a two-tailed T-test (𝑝-values) to assess the statistical significance between the double SimCEN and the best baseline model. Typically, CTR researchers consider an improvement of 0.1% in Logloss and AUC to be statistically significant [3, 50, 51, 65] be ranked higher than a randomly chosen negative one (i.e., ranking ability). It's worth noting that even a slight improvement in AUC and Logloss is meaningful in the context of CTR prediction tasks. Typically, CTR researchers consider improvements at the 0.001level (0.1%) to be statistically significant, as often highlighted in existing literature [3, 50, 51, 65] .Baselines. We compared SimCEN with some classical state-ofthe-art (SOTA) models. Given that deep CTR models often perform better, for models that have both non-DNN and DNN versions, we tend to choose the latter. The list of models we have chosen in chronological order of publication is as follows: FM [41] (2010); DNN [8] , PNN [38] , Wide & Deep [6] (2016); DeepFM [14] , DCN [51] (2017); xDeepFM [29] (2018); FiGNN [27] , AutoInt+ [43] (2019); AFN+ [7] (2020); DCNv2 [52] , EDCN [3] , MaskNet [54] (2021); GraphFM [28] (2022); CL4CTR [50] , EulerNet [44] (2023).Implementation Details. We implemented all models using Pytorch [37] and refer to existing works [21, 65] . We employ the Adam optimizer [23] to optimize all models, with a default learning rate set to 0.001. For the sake of fair comparison, we set the embedding dimension of MicroVideo and KuaiVideo to 64 [26] , and the embedding dimension of other datasets to 16 [63, 65] , the numbers of MLP hidden units are [400, 400, 400], and the batch size to 10,000 for all models. The hyperparameters of the baseline model were configured and finetuned based on the optimal values provided in [21, 65] and their original paper.We not only compared SimCEN with the selected 16 baseline models but also further investigated the joint performance of SimCEN with MLP and the performance of the double SimCEN. The overall experimental results are shown in Table 3 . We can draw the following observations:• Models based on the parallel structure (e.g., DCN, DeepFM, Au-toInt*), by decoupling feature interaction learning into parallel learning of explicit and implicit interactions, improve performance. This confirms the rationality of separately modeling explicit and implicit feature interactions. • Models based on the stacked structure (e.g., PNN, MaskNet, FiGNN) improve performance through sequential deconstruction of feature interaction learning. It is worth noting that, for example, in the cases of PNN and AutoInt*, there is no absolute advantage of stacked structure over parallel structure, vice versa. On the Avazu dataset, the former is better than the latter, while on the Criteo dataset, it is the opposite. • As depicted in Table 1 , the standard alternate structure achieves notable improvements primarily in Logloss optimization, while its performance in AUC is lacking. Nevertheless, the double Sim-CEN consistently demonstrates statistically significant enhancements in both Logloss and AUC as verified by t-tests (with 𝑝values), and SimCEN even outperforms the strongest baseline models. This serves as evidence of the efficacy of contrastive loss and intra-layer connection. In terms of AUC, SimCEN achieves an absolute gain of 0.16% on the Avazu dataset, and regarding Logloss, it achieves an absolute improvement of 0.38% on the Frappe dataset, both surpassing the 0.1% threshold for significance. This highlights the superiority of SimCEN over other complex models. • SimCEN can be jointly used with MLP or itself to capture richer feature interaction information, further enhancing performance beyond that of a single SimCEN. Meanwhile, the performance of the double SimCEN is consistently better than SimCEN + MLP, further confirming the superiority of SimCEN over MLP. More specifically, SimCEN + MLP achieves an average absolute improvement of 0.2% for Logloss and 0.15% for AUC across the six datasets, while the double SimCEN achieves an improvement of 0.61% for Logloss and 0.24% for AUC. This also demonstrates that SimCEN's optimization for Logloss is more remarkable. The results of the ablation study are illustrated in Table 5 . It can be observed that the performance degradation is more pronounced when removing the contrastive product and the alternate structure. This demonstrates the effectiveness of multiple semantic spaces and the alternate structure. Furthermore, we also note a performance loss when eliminating the contrastive loss and the intralayer connection, emphasizing the importance of balancing both homogeneity and diversity.In order to confirm the compatibility of SimCEN, we treat it as a substitute for MLP and incorporate it into other classic baseline models. The experimental results are shown in Table 4 . It is evident that relative to traditional DNN, our proposed SimCEN achieves significant improvements (greater than 0.1%) on all six datasets, with average improvements of 0.77% and 0.27% in Logloss and AUC optimization, respectively. Across all models, SimCEN brings particularly significant performance gains on the Avazu dataset, providing over 0.3% absolute improvement in AUC.In terms of Logloss optimization, SimCEN delivers a 1% absolute improvement for AFN. This demonstrates the effectiveness and compatibility of SimCEN. Additionally, by observing the average performance improvements brought by SimCEN, we can see that its optimization for Logloss is greater than the improvement in AUC, further confirming the effectiveness of the alternate structure. To explore the impact of balancing diversity and homogeneity in different semantic spaces on the final representation, we conducted a comparison between SimCEN and a simple DNN (with input augmented to three times the embeddings). We initially randomly sampled 1,000 instances from the final representations. Subsequently, we employed t-SNE [46] to map these representations into a 2-dimensional space and visualized the distribution as depicted in Figure 6 . Evidently, the final representations learned by SimCEN are dispersed, whereas the DNN obtains relatively concentrated representations. This implies that by simultaneously considering diversity and homogeneity, we can acquire richer and low-redundancy feature interaction information. In contrast, the semantic information obtained by DNN is relatively narrow. • #1: V 𝑎𝑙𝑡 𝐿 = V 𝑎𝑙𝑡 𝐿 ⊙ 𝑔(V 𝑎𝑙𝑡 𝐿 ), • #2: ego 𝐿 = ego 𝐿 ⊙ 𝑔(ego 𝐿 ); v 1 𝐿 = v 1 𝐿 ⊙ 𝑔(v 1 𝐿 ); v 2 𝐿 = v 2 𝐿 ⊙ 𝑔(v 2 𝐿 ), • #3: v 1 𝐿 = v 1 𝐿 ⊙ 𝑔(v 1 𝐿 ); v 2 𝐿 = v 2 𝐿 ⊙ 𝑔(v 2 𝐿 ), • #4: ego 𝐿 = ego 𝐿 ⊙ ego 𝐿 ; v 1 𝐿 = v 1 𝐿 ⊙ ego 𝐿 ; v 2 𝐿 = v 2 𝐿 ⊙ ego 𝐿 , • #5: ego 𝐿 = ego 𝐿 ⊙𝑔(ego 𝐿 ); v 1 𝐿 = v 1 𝐿 ⊙𝑔(ego 𝐿 ); v 2 𝐿 = v 2𝐿 ⊙𝑔(ego 𝐿 ), where 𝑔(𝑥) = 𝜎 (W𝑥 + b). The experimental results are presented in Table 6 . From Table 6 , we observe that variants #6 achieved the best results, demonstrating the superiority of external-gated over self-gated. It is worth noting that we found global gating is not always a good choice (e.g., variants #1, #2, #5), preserving some of the original representations can further improve performance (e.g., variants #2, #3 on Avazu). 
related work 5.1 deep ctr prediction: 
 Most existing deep CTR prediction models can be categorized into user behavior sequences based models [12, 15, 32, 61] and feature interaction based models [3, 14, 27, 29, 39, 48, 51, 52, 54, 60] . As SimCEN belongs to the latter category, we provide a summary of relevant feature interaction based models, which generally employ two frameworks: parallel and stacked structure. Wide & Deep [6] , DeepFM [14] , DeepLight [9] , FinalMLP [34] , and DCN [51] use parallel structures typically divide the original embeddings into two or more different semantic spaces, capturing low-order and high-order feature interactions in parallel through explicit and implicit components. On the other hand, NFM [18] , PNN [38] , MaskNet [54] , and xCrossNet [60] use stacked structures input embeddings processed by explicit components into implicit components, helping the implicit components acquire more diverse feature interactions of different orders. However, these models often attempt to propose more complex ways of feature interaction without trying to break the limitations of these two structures. In this paper, SimCEN introduces a novel alternate structure that integrates explicit and implicit components alternately, mutually promoting each other's learning capabilities and enhancing information communication.Until now, there has been limited exploration of combining contrastive learning with CTR prediction based on feature interactions. The reason for encountering this challenge is that user click behavior is multi-interest, making it difficult to construct strict positive and negative samples between feature representations. Consequently, traditional contrastive learning principles centered on alignment and uniformity cannot be readily applied to CTR. Meanwhile, due to the similarities between CTR prediction models based on user behavior sequences and NLP [24, 33] , they initially incorporate contrastive learning. MISS [15] enhances user interest representations using contrastive loss at the feature level. AQCL [36] addresses learning difficulties in representing user click history features in cold-start scenarios through AQCL loss. CL4CTR first introduces contrastive learning to feature interaction-based CTR prediction, introducing concepts of feature alignment and field uniformity. However, it proposes a contrastive module that greatly increases training costs. 
conclusion: 
 In this paper, we revisited deep CTR prediction models based on explicit and implicit feature interactions and summarized their limitations. To achieve a more finely-grained decoupling of the learning process, we introduced the alternate structure which showcases its superiority in Logloss optimization through experiments. Subsequently, to further explore the potential of the alternate structure and solve the limitations, we attempted to integrate it with contrastive learning into a simple MLP. By balancing the diversity and homogeneity of captured feature interaction information, we proposed the SimCEN model, which significantly enhanced the performance of the MLP. We conducted extensive experiments across six real-world datasets and confirmed the compatibility and effectiveness of SimCEN.  
… … …: 
 information. Click-through rate (CTR) prediction is a vital element in achieving this goal, leveraging user profiles, item attributes, and context to predict user-item interactions. Accurate CTR predictions significantly influence system profits [3, 6, 14, 52] , while also improving user satisfaction and retention through better recognition of user interests, enhancing the overall experience. The Multi-Layer Perceptron (MLP) is a backbone component widely used in deep learning, with applications across fields like computer vision (CV) [16, 45] , natural language processing (NLP) [10, 47] , and recommender systems [19, 34, 64] . However, some studies have pointed out that while MLP is proven to be a universal function approximator [20] , it still struggles to learn certain simple product operations [42] , such as the inner product. In the CTR prediction tasks, the effectiveness of product operations has been widely proven [38, 39, 41, 43] , and these operations are integrated as explicit interaction methods in most deep CTR models to break through the performance bottleneck of implicit interactions, which are typically modeled using MLPs. According to the method of integration, as illustrated in Figure 1 , explicit and implicit components can be divided into two structural types: parallel [3, 14, 29, 51, 52] and stacked structures [27, 39, 54, 60] . The parallel structure typically integrates explicit & implicit components in a parallel manner, where both components are processed separately and their results are combined at the fusion layer. On the other hand, the stacked structure serially combines explicit & implicit components, where the output of one component is fed into the next.Despite their effectiveness, current CTR models based on the aforementioned two structures have some limitations:• Lack of Information Fusion. Most models attempt to decouple multimodal feature interactions into two independent components to model low-order and high-order feature interactions simultaneously [6, 14, 51] . However, compared to the alternate structure, their components are too independent, neglecting layer-by-layer information fusion and interaction. • Feeble Communication and Supervision Signals. Typically, the augmented feature embeddings are treated as additional semantic spaces to help the model capture more diverse information [27, 34, 43] . However, these semantic spaces lack an effective means of communication and auxiliary supervision signals to prevent the model from learning redundant information.• Excessive Noise in Feature Interactions. The information gained from the traditional transition of feature interactions from low-order to high-order is not always effective, which often introduces a significant amount of noise [9, 30, 62] . Therefore, we need to seek more efficient ways of interaction.In this paper, we try to address the aforementioned limitations from the perspective of contrastive learning (CL) [22, 57] in a simple yet effective manner, exploring a feedforward neural network more suitable for CTR prediction. Recently, CL has garnered sustained interest across multiple domains [4, 13, 24, 33, 58, 59] . The CL aids model learning in a self-supervised manner to obtain higher-quality representations, where the fundamental concept involves introducing alignment and uniformity constraints [53] between samples from different views obtained through data augmentation. For CTR prediction, this idea can, with some adjustments, be extended to the capture of feature interaction information across multiple semantic spaces, enabling each space to receive auxiliary supervision signals and enhancing the model's robustness. However, we observe that most contrastive learning methods [50, 55, 56, 59] do not facilitate communication between multiple views (semantic spaces), which limits the effectiveness of the models. For another thing, most existing CTR models research endeavors to set up more complex explicit components to further enhance the model's performance [7, 29, 44, 48, 49] , while neglecting exploration into components' communication and supervision signals. Therefore, our work firstly defines the concept of alternate structure and introduces a contrastive loss into a simple MLP to address the aforementioned limitations, leading to the proposal of a new improved MLP framework, named the Simple Contrastenhanced Network (SimCEN). Overall, alternate structure implies building features interaction within the network in a way that they are intrinsically part of the model's architecture, rather than being separate components. This leads to effective multimodal information fusion. Contrastive learning refers to the use of data itself as supervision, which could provide additional signals to guide the MLP in learning richer interactions without the need for explicit labels. More specifically, SimCEN is comprised of several key ideas: (1) contrastive product, which augments the feature embeddings to obtain second-order interaction information with the same semantics but different representation spaces; (2) external-gated mechanism, which filters and interacts with the feature information across multiple semantic spaces; (3) balancing diversity and homogeneity [25] , which employs different activation functions and dropout rates across different semantic spaces of the hidden layers, and utilizes the intra-layer connection and contrastive loss (L 𝑐𝑙 ) in the final representation. The major contributions of this paper are summarized as follows:• We propose a new alternate structure, which leads to a finergrained aggregation of feature interaction information by layerby-layer integrating explicit and implicit components.  
 
 Appendix:  


Decision: To address the limitations of existing CTR models, this work introduces a novel model by integrating alternate structure and contrastive learning into only one simple MLP. Most reviewers lean towards acceptance. After reading the paper, I think the paper meets the MM standards. Thus I recommend acceptance.

Reviews: review: paper_summary: paper_summary: The paper summarizes the limitations of existing CTR methods to break through the bottleneck: Most of the current deep CTR models resort to parallel or stacked structures to break through the performance bottleneck of Multi-Layer Perceptron. The author introduces a novel model by integrating alternate structure and contrastive learning into only one simple MLP, discarding the design of multiple MLPs responsible for different semantic spaces, named the Simple Contrast enhanced Network (SimCEN),  which employs a contrastive product to build second-order feature interactions that have the same semantic but different  representation spaces. Additionally,  it employs an external-gated mechanism between linear layers to facilitate explicit learning of feature interactions. It employs an external-gated mechanism between linear layers to facilitate explicit learning of feature interactions and to filter out noise. At the final representation layer of the MLP,  a contrastive loss is incorporated to help the MLP obtain self-supervised signals for higher-quality representations.  Experiments conducted on six real-world datasets demonstrate the effectiveness and compatibility of this simple  framework, which can serve as a substitute for MLP to enhance various representative baselines.
strength_weakness: strength_weakness: 1 The author try to address the aforementioned limitations from the perspective of contrastive learning in a simple yet effective manner, exploring a feedforward neural network more suitable for CTR prediction, and the performance is better.
2 A fine-grained contrast Angle is designed, combining the fusion of implicit and display information.
limitations: limitations: 1 The paper re-examines the strategy of combining implicit and explicit paradigms, and gives Parallel structures、Stacked structures、 alternate structure. However, which types of baselines are selected? In addition, how to define the so-called explicit and implicit paradigms is not clearly given in this paper.
2 In Section 2.1, the paper presents the Multimodal Feature Embedding. In fact, the core of the multi-modal task is how to deal with the feature, the fusion feature and the gap between the modes. The thesis lacks the specific introduction of Multimodal Feature Embedding. How does the feature correspond to explicit & implicit?
3 Contrastive Product, the core component of this paper, is somewhat similar to NCF, but the introduction of part 3.1 is too vague. What is the meaning of two transformation matrices? Is there any theoretical support?
paper_summary: paper_summary: This paper presents a novel model named Simple Contrast-enhanced Network (SimCEN) for the click-through-rate prediction task. The proposed SimCEN is based on the alternate structure. In SimCEN, a contrastive product module is introduced to build second-order feature interactions. An external-gated mechanism between linear layers is used to facilitate feature interactions and filter out noise. Experiments on six datasets demonstrate the effectiveness of the proposed method.
strength_weakness: strength_weakness: * The paper is well-motivated. Section 2 described different structures for CTR prediction. Experimental results show the advantages of the alternate structure. 
* The experiments are sufficient to demonstrate the superiority of the proposed method. In particular, six datasets and more than 15 baselines are included in the experiments, and the proposed SimCEN method achieves the best performance. Ablation studies of SimCEN are conducted to further demonstrate the effectiveness of each component in SimCEN.
limitations: limitations: * Contrastive learning technology has been applied in the CTR prediction task. In line 135, the differences between the proposed method and the existing contrastive learning based methods are captured at a high level. "... do not facilitate communication between multiple views (semantic spaces)..." is too general. It will be good if the detailed comparison can be included there. 

* From Table 3 and Table 4, the SimCEN seems to be a module that can be integrated with existing models. It is not clear to me what the architectures of the "SimCEN + MLP" and "SimCEN + SimCEN" look like. 

* Fig. 2 shows the advantage of the alternate structure. The term "average distance" can be very confusing in the figure. It will be good to include the explanation of "average distance" in the caption. 

* Section 4.3.4 and Fig. 6 show the visualization of the final representation using t-SNE. It is not clear to me the meaning of the colors in the figure.
paper_summary: paper_summary: This paper proposes a novel alternate structure that achieves finer-grained aggregation of feature interaction information by integrating explicit and implicit components layer by layer. It introduces an effective contrastive learning framework that enhances the MLP's ability to model feature interactions by balancing diversity and homogeneity in representations. Comprehensive experiments on six real-world datasets demonstrate the effectiveness and compatibility of the proposed SimCEN.
strength_weakness: strength_weakness: S1. Relevance of the Problem: It addresses a crucial issue in feature interaction modeling, which is vital for improving the performance and accuracy of CTR prediction.

S2. Innovative Methodology: The introduction of a novel alternate structure for finer-grained aggregation and a simple yet effective contrastive learning framework showcases significant methodological advancements, balancing diversity and homogeneity in representations.

S3. Empirical Validation: The paper provides comprehensive experimental validation across six real-world datasets, demonstrating the proposed SimCEN's effectiveness and compatibility, reinforcing its practical applicability and robustness.
limitations: limitations: W1. Scope: The research focuses on CTR prediction and does not appear to address multimodal aspects. The six datasets used in the experiments also do not seem to relate to multimodal recommendation, making the paper more suited for conferences focused on recommendation systems.

W2. Limited Improvement: As seen in Table 3, the performance improvement of SimCEN seems modest. The fundamental difference between the alternate structure and the stacked structure is unclear. Figure 1 suggests that the alternate structure may simply add more layers of implicit interaction (MLP), implying that the improvements might be primarily due to increased layer depth, which is an expected result.

Overall, the experiment is still quite comprehensive, and I hope the authors will provide some reasonable responses if I understand uncorrectly.
paper_summary: paper_summary: this proposes a novel approach to improve click-through rate predictions using contrastive learning techniques. The method demonstrates superior accuracy and robustness compared to traditional models, offering significant benefits for online advertising and recommendation systems. The paper thoroughly discusses the methodology, results, and implications, making a strong case for the adoption of SimCEN in practical applications.
strength_weakness: strength_weakness: The paper excels in several key areas, making it a valuable contribution to the field. Its novelty, strong theoretical foundation, technical correctness, comprehensive evaluation, clarity, and practical applications collectively highlight its strengths. This work not only advances the state-of-the-art in CTR prediction by a very single design but also opens up new avenues for applying contrastive learning in other prediction tasks.
limitations: limitations: The writing should be improved. Like:
1. When you mention the limitations in the abstract, your contribution in the introduction do not talk about how to filter out noise information.

2. It is hard to know some definitions for people who do know well in this area. You have to explain what is explicit and implicit? what is the definition of communication?

3. You mentioned three limitations in the introduction, but in abstract there is only two. It is confusing.

4. You directly write the dropout rate in Fig. 4. Is it a hyperparameter, but you fix it?

For the experiment, I have some concerns:

1. 4 of the 6 datasets do not contain features like images or videos. They can not evaluate the multimodal feature interactions, which is really important in the Multimedia conference. Furthermore, in your ablation study, you choose the datasets that do not contains images or videos.

2. Some typos, like line 616, you use AutoInt+ and AFN+, but in table 3, you use AutoInt* and AFN*. 

3. In the experiment, I am confused about the advantages of the explicit and implicit components in SimCEN. Why it is better than other parallel structure? Or it is just the same?