Multi-Epoch Learning with Data Augmentation for Deep Click-Through Rate Prediction

## Abstract

Abstract. This paper investigates the one-epoch overfitting phenomenon in Click-Through Rate (CTR) models, where performance notably declines at the start of the second epoch. Despite extensive research, the efficacy of multi-epoch training over the conventional one-epoch approach remains unclear. We identify the overfitting of the embedding layer, caused by high-dimensional data sparsity, as the primary issue. To address this,
we introduce a novel and simple Multi-Epoch learning with Data Augmentation (MEDA) framework, suitable for both non-continual and continual learning scenarios, which can be seamlessly integrated into existing deep CTR models and may have potential applications to handle the “forgetting or overfitting” dilemma in the retraining and the well-known catastrophic forgetting problems.
MEDA minimizes overfitting by reducing the dependency of the embedding layer on subsequent training data or the Multi-Layer Perceptron (MLP) layers, and achieves data augmentation through training the MLP with varied embedding spaces.
Our findings confirm that pre-trained MLP layers can adapt to new embedding spaces, enhancing performance without overfitting. This adaptability underscores the MLP layers’ role in learning a matching function focused on the relative relationships among embeddings rather than their absolute positions.
To our knowledge, MEDA represents the first multi-epoch training strategy tailored for deep CTR prediction models.
We conduct extensive experiments on several public and business datasets, and the effectiveness of data augmentation and superiority over conventional single-epoch training are fully demonstrated.
Besides, MEDA has exhibited significant benefits in a real-world online advertising system. † † ∗ * ∗ Equal contributions from the three authors.

## 1. Introduction

Click-through rate (CTR) prediction is crucial in online recommendation and advertising systems, benefiting significantly from advancements in deep learning-based models (Cheng et al., ; Qu et al., ; Guo et al., ; Yu et al., ; Zhou et al., , ; Pi et al., ; Li et al., ). Despite the progress and diverse approaches, including non-continual learning for smaller datasets and continual learning (Cai et al., ; Guan et al., ; Mi et al., ; Yang et al., ) for larger or real-time datasets, a common challenge persists: “one-epoch overfitting” (Zhou et al., ; Zhang et al., ).
This phenomenon, where model performance drops sharply at the beginning of the second training epoch, contrasts with other deep learning domains like computer visionn (He et al., ; Russakovsky et al., ), audio processing (Purwins et al., ), and natural language processing (Devlin et al., ; Vaswani et al., ), where multiple epochs enhance model convergence.
Specifically, for industrial areas with sufficient computation resources and confronted with model-convergence issues (e.g., with small datasets), multi-epoch learning is valuable. Furthermore, multi-epoch learning allows necessary “rethinking”, exemplified in tasks like unsupervised domain adaptation (Wilson and Cook, ) or label-noise correction (Song et al., ). Therefore, addressing the one-epoch overfitting issue is crucial for improving the efficacy of CTR prediction models in industrial settings.

Unfortunately, most research (Belkin et al., ; Salman and Liu, ; Zhou, ; Zhang et al., ; Arpit et al., ) on the overfitting problem of Deep Neural Networks (DNNs) has predominantly focused areas outside CTR prediction.
Moreover, to our knowledge, in addition to the CTR domain, only large language models undergoing supervised fine-tuning have reported similar one-epoch overfitting phenomena (Ouyang et al., ; Komatsuzaki, ).
Despite recent efforts (Zhang et al., ) to dissect the underlying factors of this phenomenon within deep CTR models, the potential benefits of adopting a multi-epoch training paradigm remain unexplored, as optimal performance is often attained with a single epoch of training.

Figure: Figure 1. Our proposed MEDA framework. For non-continual learning, MEDA reinitializes the embedding parameters at the onset of each training epoch; for continual learning, MEDA maintains multiple independently initialized embedding layers and for each dataset, trains each embedding layer once successively. The embedding layers can be selected based on requirements or costs.
Refer to caption: extracted/5694636/pic/framework5.png

To address the one-epoch overfitting challenge in CTR models, we introduce a novel and simple Multi-Epoch learning with Data Augmentation (MEDA) framework, tailored for both non-continual and continual learning scenarios. Our framework can also cover both the classification and regression tasks. Specifically, we identify the overfitting of the embedding layer, caused by high-dimensional data sparsity, as the primary issue for one-epoch overfitting, then design MEDA to effectively mitigate overfitting by decoupling the embedding and Multi-Layer Perceptron (MLP) layers in CTR models (Zhang et al., ), addressing the core issues of high data dependency and low occurrence rates in embedding vectors that lead to poor generalization on test datasets, and reducing the embedding-data and embedding-MLP dependencies. In detail, in our non-continual MEDA algorithm shown in Figure 1, the embedding-data dependency is reduced by reinitializing the embedding layer at the onset of each training epoch. The non-continual MEDA is extended to the continual MEDA to further reduce the additional embedding-MLP dependency in the continual learning setting, which is shown in Figure 1. We leverage multiple independently initialized embedding layers—each for an extra epoch: for each dataset, each embedding layer can be selected to train once successively. The selection can be based on business requirements or training costs. Various variants of our continual MEDA can be designed for specific requirements. Intuitively, on each dataset, each embedding layer in MEDA is trained once only, thereby minimizing overfitting, while the MLP layers are trained repeatedly to improve convergence. Our proposed MEDA can be regarded as a data augmentation method because it can be treated as learning the same categorical features with different embedding spaces. To our knowledge, MEDA represents the first multi-epoch training strategy tailored for deep CTR prediction models. Potential applications of our MEDA may be designed to handle the “forgetting or overfitting” dilemma in the *retraining* (Wang et al., ) and the well-known *catastrophic forgetting* (Katsileros et al., ; Hu et al., ) problems, in addition to the normal multi-epoch training task.

We conduct comprehensive experiments on public and business datasets to show the effectiveness of data augmentation and superiority over single-epoch learning.
Notably, MEDA’s second-epoch performance consistently exceeds that of single-epoch training across various datasets and CTR models, with improvements in test AUC ranging from 0.8% to 4.6%. This trend persists across multiple epochs without inducing overfitting, offering flexibility in training duration based on cost considerations. Our findings confirm that pre-trained MLP layers can adapt to new embedding spaces, enhancing performance without overfitting. This adaptability underscores the MLP layers’ role in learning a matching function focused on the relative relationships among embeddings rather than their absolute positions. Furthermore, MEDA demonstrates remarkable efficiency by achieving or surpassing the outcomes of complete data single-epoch training with only a fraction of the data, e.g., in most cases, MEDA with $1/2$ data can outperform single-epoch training with complete data, sometimes even $3$ epochs on $1/8$ data can outperform $1$ epoch on complete data, indicating its potential for significant data augmentation benefits.
The successful deployment of MEDA in a live environment, corroborated by positive online A/B testing results, further attests to its practical value and impact.

## 2. Related Works

*One-Epoch Overfitting*.
Recent empirical research (Zhang et al., ) indicates that reducing feature sparsity can diminish the prevalence of one-epoch overfitting, yet the potential superiority of multi-epoch training over traditional single-epoch approaches remains uncertain. Additionally, established anti-overfitting strategies such as weight decay and dropout have proven ineffective in this context (Zhang et al., ). Parallel observations (Ouyang et al., ) in large language models undergoing supervised fine-tuning reveal a similar tendency towards one-epoch overfitting, albeit with a suggestion that a moderate level of overfitting might actually benefit downstream tasks. This concept of ”appropriate overfitting” presents an intriguing avenue for future exploration within our proposed framework.

*Pre-training*.
Recent approaches (Lin et al., ; Liu et al., ; Wang et al., ; Muhamed et al., ) have explored pre-training to enhance the representational capabilities of embedding and feature extraction layers within MLPs for various applications, yet these advancements fall short in demonstrating their efficacy in avoiding overfitting when CTR prediction is incorporated as an auxiliary training objective, nor do they facilitate multi-epoch training for such models. In contrast, graph learning research has delved into fine-tuning pre-trained models for new graphs, facing challenges related to either maintaining a consistent node ID space (Hu et al., ; Liu et al., ; Lu et al., ; Hu et al., ) or solely leveraging graph structure while neglecting node features (Qiu et al., ; Zhu et al., ). This leaves an open question in the context of CTR models: the potential for pre-trained MLP layers to contribute positively to a distinct embedding space remains unexplored and warrants further investigation.

## 3. Background

*Features*.
CTR (Click-Through Rate) prediction models are distinguished by their handling of high-dimensional sparse data, often involving billions of categorical features with low occurrence rates (Jiang et al., ; Zhao et al., , ). These features generally encompass (Cheng et al., ; Qu et al., ; Guo et al., ; Yu et al., ; Zhou et al., , ; Pi et al., ; Li et al., ) item profiles (including Item ID, brand ID, shop ID, category ID, etc.), user profiles (User ID, age, gender, income level), and both long- and short-term user behaviors, reflecting a mix of enduring interests and immediate needs. Prior to modeling, raw features undergo discretization (Liu et al., ) to transform numerical inputs into categorical ones and feature selection (Guo et al., ) to refine the dataset, resulting in predominantly categorical inputs. In real-world industrial applications, categorical features are typically extremely high-dimensional sparse (Jiang et al., ; Zhao et al., , ).

*Embedding*.
To handle these categorical features, most deep CTR prediction models share a similar Embedding and Multi-Layer Perceptron (MLP) architecture.
Specifically, they typically adopt an embedding layer (Zhang et al., ) at the front, followed by various types of MLP structures, with the embedding layer responsible for mapping the high-dimensional categorical features to low-dimensional vectors.

*MLP*.
Given the concatenated dense representation vector, an MLP is employed to capture the nonlinear interaction among features (Liu et al., ).
Numerous studies (Cheng et al., ; Qu et al., ; Guo et al., ; Yu et al., ) have concentrated on devising MLP architectures that excel in extracting pertinent information for tabular data.
It is worth noting that, given the rapid expansion of user historical behavior data in real-world industrial applications, careful attention must be paid to user behavior modeling during the design of MLPs.
The relevant methods (Zhou et al., , ; Pi et al., ; Li et al., ) considering user behavior modeling focus on capturing the dynamic nature of user interests based on their historical behaviors which typically consist of lists of categorical features.

## 4. Methodology

In this section, we present the detailed methodology of our method. First, we define the notations and problem settings of our study.

*Non-continual Learning*.
Consider a dataset $\mathcal{D}=\{(\mathbf{x}^{i},\mathbf{y}_{i})\}_{i=1}^{n}$ consisting of $n$ independent samples. For the $i$th sample, $\mathbf{x}^{i}\in\mathcal{X}\subset\mathbb{R}^{d}$ is a feature vector with $d$ dimensions, $y^{i}\in\mathcal{Y}\subset\mathbb{R}^{m}$ is the label of the $i$th sample. Note that our framework can cover both classification and regression tasks. Let $M\in\mathcal{M}$ be a data-driven model. Specifically, let $\theta$ be the collection of training parameters of the MLP layers, and $\mathbf{E}$ be the collection of training parameters of the embedding layer. Let $A\in\mathcal{A}$ be a training algorithm, and we denote by $M=A(\{\mathcal{D}:k\})$ as obtaining the model $M$ by training the dataset $\mathcal{D}$ by algorithm $A$ for $k\in\mathbb{Z}_{+}$ epochs. And we denote by $S(\mathcal{D}\mid M)\in\mathbb{R}$ as the evaluation score (the bigger the better) obtained from evaluating $M$ on $\mathcal{D}$. Finally, splitting $\mathcal{D}$ into $\mathcal{D}_{tr},\mathcal{D}_{te}$ as the training and testing datasets, respectively, our goal is to see if there exists a $k>1$ such that $S(\mathcal{D}_{te}\mid A(\{\mathcal{D}_{tr}:k\}))>S(\mathcal{D}_{te}\mid A(\{
\mathcal{D}_{tr}:1\}))$.

*Continual Learning*.
Following the notations in the non-continual learning setting, consider the training dataset $\mathcal{D}_{tr}=\{\mathcal{D}_{tr}^{t}\}_{t=1}^{T}$ consisting of $T$ successive sub-datasets. Our goal is to see if there exists a $k^{t}>1$ such that $S(\mathcal{D}_{te}\mid A(\{\mathcal{D}_{tr}^{t}:k^{t}\}_{t=1}^{T}))>S(\mathcal
{D}_{te}\mid A(\{\mathcal{D}_{tr}^{t}:1\}_{t=1}^{T}))$, where we denote by $A(\{\mathcal{D}_{tr}^{t}:k^{t}\}_{t=1}^{T})$ as training each $\mathcal{D}_{tr}^{t}$ for $k^{t}$ epochs and denote by $A(\{\mathcal{D}_{tr}^{t}:1\}_{t=1}^{T})$ as training each $\mathcal{D}_{tr}^{t}$ for $1$ epoch. The order of sub-datasets for training is not constrained.

In the following, we first introduce the modeling of and rationale for our MEDA framework, then detail two instantiations to cover the non-continual and continual learning settings, respectively.

### 4.1. Our MEDA Framework

As introduced, for multi-epoch training, we propose MEDA to simultaneously maintain feature sparsity and avoid the one-epoch overfitting.

Overfitting typically arises from a strong dependency between the training data and model parameters that fails to generalize to unseen data. Addressing overfitting involves either reducing this dependency or enhancing model generalization. In the context of CTR models, by decoupling the embedding from the MLP layers, we identify two main types of dependencies: embedding-data and embedding-MLP dependencies, where the latter views the low-dimensional embeddings as input data for the MLP layers. Our detailed experiments in Section 5 demonstrate that overfitting predominantly stems from the embedding-data interaction. Specifically, repeated training of the embedding layer on the same dataset leads to significant overfitting. In contrast, multiple training iterations over the same embedding vectors for the MLP layers result in much less or no overfitting.
This discrepancy is likely due to the sparse nature of high-dimensional data, where a vast number of categorical values exist but each appears infrequently. Consequently, embedding vectors, representing these infrequent values, are prone to overfitting due to limited training samples. Meanwhile, MLP parameters, engaging with the entire dataset, exhibit a lower risk of overfitting.

*Non-continual Problem Formulation*.
Building on the findings discussed, we approach the problem with the hypothesis that overfitting occurs when the initial parameters of the model align too closely with the exact information present in the training data. This insight leads us to the conclusion that optimizing initial parameter settings is crucial for mitigating overfitting. Notably, the phenomenon of one-epoch overfitting, which emerges at the beginning of the second training epoch, underscores that overfitting is triggered when the initial parameters precisely mirror the information within a specific data sample. To circumvent this, the initial parameters must be devoid of any exact information from future training samples. A straightforward solution is to randomize the initial parameters, ensuring their independence from subsequent training data. Given our analysis pinpointing embedding overfitting as the primary concern, we propose the novel strategy of randomly initializing embedding parameters at the start of each epoch in our non-continual MEDA framework. See Algorithm 1 for details.

Conventional wisdom might suggest that reinitializing embedding parameters could disrupt the MLP’s ability to recognize embeddings, potentially undermining training and prediction efficacy. However, our experimental findings detailed in Section 5 reveal a different outcome: the MLP not only successfully identified the embeddings but also yielded superior prediction accuracy. This indicates that the MLP was able to discern the relative relationships among embeddings. Despite significant differences in the final embedding parameters across epochs, as shown in Section 5, the essential insight is that for CTR model MLP layers, the precise values or absolute positions of embeddings are less critical than their interrelations. This understanding allows us to view the additional embeddings processed by the MLP as augmented data samples. These samples maintain crucial semantic relationships within varying embedding spaces, effectively serving as a form of data augmentation. Furthermore, our results demonstrate that the MLP was nearing convergence throughout the multi-epoch training, suggesting that despite initial embedding values differing between epochs, the semantic essence crucial for CTR prediction was consistently captured in our multi-epoch learning framework.

Figure: Algorithm 1 Non-continual MEDA

*Continual Problem Formulation*.
In a continual learning framework, where datasets are processed successively, we encounter a unique challenge: one-epoch overfitting also occurs upon the second training of the $t$th dataset $t>1$, involving both embedding and MLP layer optimization. This scenario diverges from the non-continual setting, as reinitializing embedding parameters at the start of the $t$th dataset’s training effectively disregards the accumulated knowledge from datasets $1$ to $(t-1)$, which is undesirable. Drawing from insights in the non-continual setting, to prevent one-epoch overfitting,
the initial embedding parameters should not contain *exact* information in any data sample in the $t$th dataset, but should contain information in datasets $1\sim(t-1)$.
Therefore, one option involves adopting the final embedding parameters from the $t-1$th dataset’s training $\mathbf{E}^{t-1}$. Nonetheless, given that the current MLP layers $\mbox{\boldmath$\theta$}^{c}$ have already interacted with $\mathbf{E}^{t-1}$ during the $t$th dataset training, a high dependency exists between $\mbox{\boldmath$\theta$}^{c}$ and $\mathbf{E}^{t-1}$, heightening the risk of overfitting, as our experiments in Section 5 corroborate, even if we fix $\mathbf{E}^{t-1}$ for training.

Therefore, we further propose to reduce the dependency between $\mbox{\boldmath$\theta$}^{c}$ and the initial embedding parameters for the second-epoch training of dataset $t$, which belongs to the second type of data-parameter dependency. Specifically, a group of embedding parameters with a different embedding space and trained on datasets $1\sim(t-1)$ may satisfy the constraints above. Thus we combine the solution in non-continual MEDA to independently initialize multiple groups of embedding parameters to form distinct embedding spaces. These are then trained sequentially with the MLP on each dataset. See Algorithm 2 for details.
Note that our non-continual and continual MEDA algorithms are consistent: our non-continual can be regarded as a special case of our continual MEDA for a singular dataset scenario.

Comparing our non-continual MEDA and continual MEDA for the same training and testing dataset, i.e., continual MEDA will split the training datasets, the non-continual MEDA exhibits intuitive superiority because the differences between different groups of embedding parameters are larger than those in the continual MEDA.
Consider an extreme situation of continual MEDA such that each data sample is a dataset, since the values of initial embedding parameters are usually small, the gradients for different groups of embedding parameters may be similar, which will result in similar embedding parameters. This similarity could diminish the intended effect of data augmentation achieved by training the MLP with diverse embedding spaces, which is also supported by the results in Section 5.

Figure: Algorithm 2 Continual MEDA

### 4.2. Potential Applications

*Retraining*.
Retraining requirements (Wang et al., ) arise when there are significant alterations in data structure, features, or models, necessitating model updates to align with the revised pipeline. This scenario presents a ”forgetting or overfitting” dilemma regarding the transfer of optimized model parameters from the old to the new model. Forgetting valuable information is a risk if parameters are discarded, whereas transferring them for retraining could lead to overfitting. Our non-continual MEDA approach addresses this challenge by transferring MLP for retraining to mitigate overfitting.

*Catastrophic Forgetting*.
The well-known issue of catastrophic forgetting (Katsileros et al., ; Hu et al., ) in CTR prediction represents a decline in a model’s ability to accurately predict outcomes for specific IDs not recently included in training. This phenomenon poses a significant challenge for maintaining accurate predictions on historical data. The “forgetting or overfitting” dilemma in this problem involves deciding whether to retrain the model using data from certain important IDs. The overfitting will also occur when retraining the data. Therefore, our continual MEDA may be modified for such retraining and then avoid overfitting.

### 4.3. Computation/Storage Complexity Analyses

Since our method only adds initialization processes, which are negligible for the computation complexity of training. Therefore, our method adds negligible computation complexity compared with standard multi-epoch learning. On the other hand, for the continual learning setting, our method requires $\mathcal{O}(kND)$ storage resources to maintain $k$ groups of embedding parameters for $k$-epoch learning, with $N$ representing the number of IDs and $D$ embedding vector dimension.

## 5. EXPERIMENTS

In this section, we present the experimental setup and conduct extensive experiments to evaluate the effectiveness and superiority of our proposed MEDA framework, along with ablation studies and online A/B test results.

### 5.1. Experimental Setup

**Table 1. Statistics of four datasets.**
|  | #Records | #users | #items | #categories |
| --- | --- | --- | --- | --- |
| Amazon | 51M | 1.5M | 2.9M | 1252 |
| Taobao | 89M | 1M | 4M | 9407 |
| SVO | 1.75B | 0.2B | 6M | 1259 |
| SVSL | 0.3B | 40M | 0.5M | 324 |

*Datasets*.
We conduct comprehensive evaluations on two public datasets and two business datasets, with statistics in Table 1.

Amazon dataset(^1^11[https://nijianmo.github.io/amazon/index.html](https://nijianmo.github.io/amazon/index.html)). It is a frequently used *public* benchmark that consists of product reviews and metadata collected from Amazon (Ni et al., ). In our study, we adopt the Books category of the Amazon dataset. We predict whether a user will review an item.

Taobao dataset(^2^22[https://tianchi.aliyun.com/dataset/649](https://tianchi.aliyun.com/dataset/649)). It is a *public* compilation of user behaviors for CTR prediction from Taobao’s recommender system (Zhu et al., ).

Short-Video Order (SVO) dataset. It is our collected *large business* dataset that consists of user behaviors from Kuaishou’s recommender system.
For this dataset, we predict the order behaviors of each user. We split 6 days for training and 1 day for testing.

Short-Video Search LTV (SVSL) dataset. It is also our collected *business* dataset that consists of user behaviors from Kuaishou’s *search* system.
For this dataset, we predict the Life-Time Value (LTV) (Theocharous et al., ) value of each order for each user. We split 370 days for training and 1 day for testing.

For continual learning, for both the public and the SVO datasets, we split the first half of training data as $\mathcal{D}_{tr}^{1}$ and the rest as $\mathcal{D}_{tr}^{2}$, while for the SVSL dataset, we split the first 280 days of training data as $\mathcal{D}_{tr}^{1}$ and the rest as $\mathcal{D}_{tr}^{2}$.

*CTR Models and Metrics for Evaluation*.
We apply our method on the following CTR Models:

- •
DNN is a base deep CTR model, consisting of an embedding layer and a feed-forward network with ReLU activation.
- •
DIN (Zhou et al., ) proposes an attention mechanism to represent the user interests w.r.t. candidates.
- •
DIEN (Zhou et al., ) uses GRU to model user interest evolution.
- •
MIMN (Pi et al., ) proposes a memory network-based model to capture multiple channels of user interest drifting for long-term user behavior modeling.
- •
ADFM (Li et al., ) proposes an adversarial filtering model on long-term user behavior sequences.

For the business datasets, we adopt DIN as default. We denote our non-continual and continual MEDA as MEDA-NC and MEDA-C, respectively. Our methods equal single-epoch learning when the number of epoch is $1$.
For binary classification tasks, i.e., click or order prediction, we use Area under the curve (AUC) and binary cross-entropy loss as evaluation metrics, while for the regression tasks, i.e., the LTV prediction, we use AUC score between the LTV prediction scores and the binary order labels as the evaluation metric, following the common business practice.

*Implementation Details*.
All CTR Models adhere to the optimal hyperparameters reported in their respective papers.
For public datasets, we adopt Adam (Kingma and Ba, ) as the optimizer with a learning rate of 0.001, and Glorot (Glorot and Bengio, ) as the initializer for embedding parameters.
For business datasets, we adopt Adagrad (Duchi et al., ) as the optimizer with a learning rate of 0.01, and uniform initializer with the range of 0.01.

### 5.2. Effectiveness and Superiority Evaluation

*Problem Justification*.
In Figure 2 highlights the presence and substantial impact of one-epoch overfitting. In both the Amazon and Taobao datasets, the test AUC rapidly declines starting from the second epoch of the direct multi-epoch learning. Whereas our MEDA can effectively improve the test AUC with the increase of epoch without overfitting. The overfitting issue is more pronounced in the Amazon dataset due to its higher data sparsity (less data, more IDs).

Figure: (a) Amazon
Refer to caption: x1.png

*Superiority Evaluation*.
Tables 2 and 3 highlight the significant superiority of our MEDA approach over conventional single-epoch learning. Our non-continual and continual MEDA methods outperform single-epoch learning on both public and business datasets by a substantial margin, which aligns with the improvement magnitude of each CTR model. Furthermore, continual MEDA slightly outperforms non-continual MEDA on the Taobao dataset but slightly underperforms on the Amazon, SVO, and SVSL datasets. This suggests that data augmentation in continual MEDA is weaker, and data sparsity is more severe in these three datasets. The results presented for
$2$ epochs of MEDA are reasonable for most industrial applications, considering computation and storage costs. Additionally, Figure 3 demonstrates stable increases in test AUCs for most models as the number of epochs increases. Hence, it is feasible to determine the stopping point at any epoch, as the AUC does not significantly decrease after a certain number of epochs. This user-friendly feature enables users to select the number of epochs based on training costs. Moreover, our MEDA approach outperforms single-epoch learning even more significantly when trained for
$8$ epochs, further highlighting its superiority.

**Table 2. The test AUC performance on the public datasets. MEDA methods run $2$ epochs.**
|  |  | DNN | DIN | DIEN | MIMN | ADFM |
| --- | --- | --- | --- | --- | --- | --- |
| Amazon | Single-Epoch | 0.8355 | 0.8477 | 0.8529 | 0.8686 | 0.8428 |
| MEDA-NC | 0.8450 | 0.8617 | 0.8602 | 0.8861 | 0.8507 |  |
| Improv. | +0.95% | +1.4% | +0.73% | +1.75% | +0.79% |  |
| MEDA-C | 0.8446 | 0.8588 | 0.8587 | 0.8832 | 0.8516 |  |
| Improv. | +0.91% | +1.11% | +0.58% | +1.46% | +0.88% |  |
| Taobao | Single-Epoch | 0.8714 | 0.8804 | 0.9032 | 0.9392 | 0.9462 |
| MEDA-NC | 0.9034 | 0.9265 | 0.9262 | 0.9500 | 0.9568 |  |
| Improv. | +3.2% | +4.61% | +2.3% | +1.08% | +1.06% |  |
| MEDA-C | 0.9054 | 0.9321 | 0.9281 | 0.9565 | 0.9549 |  |
| Improv. | +3.40% | +5.17% | +2.49% | +1.73% | +0.87% |  |

**Table 3. The test AUC performance on the business datasets. MEDA methods run $2$ epochs.**
|  | Short-Video Order | Short-Video Search LTV |
| --- | --- | --- |
| Single-Epoch | 0.8489 | 0.8184 |
| MEDA-NC | 0.8522 | 0.8248 |
| Improv. | +0.33% | +0.64% |
| MEDA-C | 0.8513 | 0.8233 |
| Improv. | +0.24% | +0.49% |

Figure: (a) Amazon
Refer to caption: x3.png

*Effectiveness of Data Augmentation*.
Figures 4 and 5 illustrate the behavior of MEDA during training, where the use of MEDA results in a gradual decrease in training loss from the second epoch onwards, similar to encountering new data. At the beginning of each epoch, there is a slight increase in training loss, resembling the performance when facing new data from a different domain, leading to recognition errors. Furthermore, Tables 4 and 5 demonstrate that MEDA achieves comparable test AUC to single-epoch learning with fewer data. In most cases, MEDA with half the data surpasses single-epoch training with complete data, especially on Taobao, which validates the efficacy of data augmentation. The relatively better performance on Taobao compared to Amazon suggests that Amazon exhibits more severe data sparsity, resulting in weaker performance when joining multiple samples. Notably, in the case of ADFM on Taobao, even $3$ epochs with $1/8$ of the data outperform a single epoch with complete data. This may be attributed to ADFM’s extensive behavior window and increased interactions between ID features, as MEDA enhances the importance of ID relationships, providing more opportunities for improvements.

Figure: Figure 4. The training metric curves of training DNN on the Taobao dataset, with or without non-continual MEDA.
Refer to caption: x5.png

Figure: Figure 5. The test metric curves of training DNN on the Taobao dataset, with or without non-continual MEDA.
Refer to caption: x6.png

**Table 4. The numbers of training epochs required for non-continual MEDA on the Taobao dataset with different data keeping rates of $\rho$s to achieve the test AUC with one-epoch on the complete data and the corresponding test AUCs.**
| $\rho$ |  | DNN | DIN | DIEN | MIMN | ADFM |
| --- | --- | --- | --- | --- | --- | --- |
| 100% | #Epochs | 1 | 1 | 1 | 1 | 1 |
| Test AUC | 0.8714 | 0.8804 | 0.9032 | 0.9392 | 0.9462 |  |
| 50% | #Epochs | 2 | 2 | 3 | 3 | 2 |
| Test AUC | 0.8864 | 0.8989 | 0.9139 | 0.9444 | 0.9525 |  |
| 25% | #Epochs | 4 | 3 | 6 | 13 | 2 |
| Test AUC | 0.8802 | 0.8847 | 0.9048 | 0.9395 | 0.9466 |  |
| 12.5% | #Epochs | 7 | 7 | 16 | $16^{*}$ | 3 |
| Test AUC | 0.8716 | 0.8844 | 0.9030 | 0.9287 | 0.9470 |  |

**Table 5. The same experiment as that conducted in Table 4 on the Amazon dataset.**
| $\rho$ |  | DNN | DIN | DIEN | MIMN | ADFM |
| --- | --- | --- | --- | --- | --- | --- |
| 100% | #Epochs | 1 | 1 | 1 | 1 | 1 |
| Test AUC | 0.8355 | 0.8477 | 0.8529 | 0.8686 | 0.8428 |  |
| 50% | #Epochs | 10 | 4 | $16^{*}$ | 4 | 7 |
| Test AUC | 0.8370 | 0.8551 | 0.8481 | 0.8879 | 0.8441 |  |
| 25% | #Epochs | $16^{*}$ | $16^{*}$ | $16^{*}$ | $16^{*}$ | $16^{*}$ |
| Test AUC | 0.8268 | 0.8446 | 0.8337 | 0.8578 | 0.8319 |  |
| 12.5% | #Epochs | $16^{*}$ | $16^{*}$ | $16^{*}$ | $16^{*}$ | $16^{*}$ |
| Test AUC | 0.8093 | 0.8262 | 0.8157 | 0.8328 | 0.8202 |  |

*Effectiveness of MLP Convergence*.
In Figure 6 (a) and (b), our MEDA approach is shown to enhance MLP convergence, as evidenced by the increasing similarity between two sets of MLP parameters in successive epochs. The cosine similarity between parameter groups continues to rise with each epoch, while the $\ell_{2}$ distance ceases to decrease after epoch 6, indicating that parameter direction is more crucial for CTR models than parameter distance. Moreover, the substantial discrepancy in final embedding parameters even at epoch 15 underscores the data augmentation effect of MEDA. Interestingly, the embedding parameters exhibit convergence on Taobao based on cosine similarity but not notably on Amazon, possibly due to the challenges posed by severe data sparsity in learning similar embedding patterns across epochs.

Figure: (a) Cosine Between MLPs
Refer to caption: x7.png

### 5.3. Ablation Study

Due to the space limit, some ablation studies are conducted on the Amazon dataset only, for whose overfitting issue is more severe.

*Primary Responsibility of Embedding*.
In Figure 7, we show that the primary causative factor of one-epoch overfitting is the embedding overfitting, where we compare $4$ groups of variants:

- (1)
Emb (MLP) fix: fixing the embedding (MLP) parameter and training the MLP (embedding).
- (2)
Emb (MLP) 1 epoch fix: after epoch $1$, fixing the embedding (MLP) and training the MLP (embedding);
- (3)
Emb (MLP) same init: for each epoch, using the same initialization result to reinitialize the embedding (MLP);
- (4)
Emb (MLP) Reinit: for each epoch, independently reinitialize the embedding (MLP).

The group “Emb (MLP) fix” reveals that the MLP does not overfit when the embedding is fixed, even containing considerable noise due to being solely initialized. Across all variants, severe overfitting occurs unless the embedding is controlled. When the embedding is controlled, overfitting can be mitigated (group 2) or avoided altogether (groups 1, 3, and 4). These findings underscore that one-epoch overfitting primarily stems from embedding overfitting.

Figure: Figure 7. The test AUC curves for training DNN on the Amazon dataset with different training paradigms.
Refer to caption: x11.png

*Variants of Continual MEDA*.
In Figure 8, we test two variants of our Continual MEDA:

- (1)
Data 1 Emb as Initial: using the final embedding of training $\mathcal{D}_{tr}^{1}$ as the initial embedding for multi-epoch training of $\mathcal{D}_{tr}^{2}$.
- (2)
Data 1 Emb as Fixed: after training $\mathcal{D}_{tr}^{2}$, using and fixing the final embedding of training $\mathcal{D}_{tr}^{1}$ for multi-epoch training of $\mathcal{D}_{tr}^{2}$.

Both variants exhibit overfitting beyond epoch 10, indicating that when the MLP is trained repeatedly with the same embedding trained on data, the dependency between them intensifies, resulting in overfitting. This highlights the significance of our continual MEDA approach in reducing the dependence between the embedding and the MLP.

Figure: Figure 8. The test AUC curves of DNN on the Amazon dataset, comparing different variants of continual MEDA to train $\mathcal{D}_{tr}^{2}$ multiple times. The continual MEDA has run 2, 4, 8, 16, and 32 epochs. Note that the results of MEDA methods are only for reference because they also train $\mathcal{D}_{tr}^{1}$ multiple times.
Refer to caption: x12.png

We also test two variants of our Continual MEDA on four datasets in Table 6:

- (1)
MEDA-C Emb-Reuse: same with the “Data 1 Emb as Initial”.
- (2)
MEDA-C Multi-MLP: same with continual MEDA, except for that when training $\mathcal{D}_{tr}^{1}$ for $\mathbf{E}_{1}^{2}$, we reinitialize the MLP.

The results show that, compared with continual MEDA, both training $\mathcal{D}_{tr}^{1}$ only once and using another MLP’s embedding compromise the performance. Nonetheless, both variants still outperform “Single-Epoch”, showing the effectiveness of multi-epoch learning.

**Table 6. The test AUC results on four datasets, comparing different variants of continual MEDA to train $\mathcal{D}_{tr}^{2}$ multiple times. Except for “Single-Epoch”, other methods run $2$ epochs.**
|  | Amazon | Taobao | SVO | SVSL |
| --- | --- | --- | --- | --- |
| Single-Epoch | 0.8355 | 0.8714 | 0.8489 | 0.8184 |
| MEDA-NC | 0.8450 | 0.9034 | 0.8522 | 0.8248 |
| MEDA-C | 0.8446 | 0.9054 | 0.8517 | 0.8233 |
| MEDA-C Emb-Reuse | 0.8397 | 0.8943 | 0.8512 | 0.8221 |
| MEDA-C Multi-MLP | 0.8382 | 0.8939 | 0.8502 | 0.8213 |

*Hyperparameter Robustness*.
In Figure 9, we examine the impact of reversing the embedding order or omitting some (even or odd) groups of embeddings in training
$\mathcal{D}_{tr}^{2}$
using our continual MEDA. Both types of variants result in compromised performance, indicating that either training on old embeddings or reducing data augmentation is inferior. Nonetheless, these variants still demonstrate enhanced performance and robustness of our MEDA methodology across increasing epochs without overfitting.

Figure: Figure 9. The test AUC curves with DNN on the Amazon dataset, comparing different variants of our continual MEDA.
Refer to caption: x13.png

### 5.4. Online Result

We conduct an online A/B test on Kuaishou’s industrial video advertising platform, focusing on retention prediction. This platform processes billions of user requests daily, with millions of item candidates. It incorporates a four-stage recommender system: candidate retrieval, pre-ranking, ranking, and reranking, each progressively narrowing down the item selection for users. We deploy MEDA in the ranking module. The baseline method performs single-epoch learning, while we adopt our continual MEDA to conduct the online A/B experiment.
The experiment spanned 9 days, with 10% of the total online traffic allocated for both the baseline and our MEDA approach. Results in Table 7 demonstrate that MEDA significantly enhances the test AUC, user retention, and *overall* platform rewards (as evaluated by revenue and revenue expected by clients). This marks the first solution addressing overfitting in multi-epoch training of large-scale sparse models for advertising recommendations. In our scenario, achieving satisfactory model performance typically necessitates training on one month’s worth of data. Remarkably, with MEDA, comparable results can be obtained after just two weeks of training. Thus, implementing MEDA substantially reduces the required sample size and training costs while maintaining equivalent outcomes.

**Table 7. Online A/B Test Performance.**
|  | Test AUC | Retention | Revenue | Expected Revenue |
| --- | --- | --- | --- | --- |
| Improv. | +0.14% | +6.6% | +0.32% | +0.91% |

## 6. Discussion

One limitation of this study is the use of independent random embedding initializations for multiple epochs, potentially discarding valuable information in the final embedding parameters from the first epoch. For instance, certain high-frequency categorical features like item category IDs may not be at risk of overfitting, suggesting that their corresponding embedding parameters should be retained. However, a potential solution exists to address this issue. Upon closer examination of our MEDA framework, a key constraint is that initial parameters must not contain *exact* information from any data sample in upcoming training data. This constraint aligns with principles of differential privacy (Chaudhuri et al., ; Dwork et al., ), where the goal is to limit the inference of *exact* information from trained parameters. By treating the random initialization process as noise addition, tailored noise levels can be designed based on the data frequency of each categorical feature. This approach could enhance MEDA’s performance further, offering an avenue for future exploration.

Conventional wisdom might suggest that multi-epoch learning could sacrifice timeliness in pure online learning (Burlutskiy et al., ). Yet, it is a shared problem of all multi-epoch learning approaches and is mainly due to limited computation resources. Recent industrial advances combine batch and online learning to achieve both accuracy and timeliness, then multi-epoch learning including MEDA can contribute to batch learning to meet resource constraints.

## 7. Conclusion

In this paper, we propose a novel and simple Multi-Epoch learning with
Data Augmentation (MEDA) framework, covering both non-continual and continual learning settings.
The experimental results of both public and business datasets show that MEDA effectively achieves the desired effect of data augmentation and multi-epoch learning can outperform
the conventional single-epoch training by a significant margin.
Furthermore, MEDA’s deployment in a real-world online advertising system and subsequent A/B testing demonstrate its substantial benefits in practical applications.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: please check the weaknesses.
soundness: 2
strengths: 1,Mitigates OEO Phenomenon: By reinitializing embedding parameters at each training epoch, the proposed approach (MEDA) effectively mitigates the Over-fitting Embedding Overhaul (OEO) phenomenon, leading to enhanced performance in CTR prediction models.

2,Thorough Analysis of Problem Causes: The paper provides a detailed analysis of the underlying causes of the embedding overfitting issue, showing a strong grasp of the challenges inherent to CTR prediction.

3,Promising Experimental Results: Results across multiple datasets demonstrate the proposed method’s effectiveness, adding empirical support to its benefits.

4,Clarity of Presentation: The clear and well-organized writing style makes the paper easy to follow, especially for readers unfamiliar with the nuances of CTR modeling.
summary: This paper investigates OEO, considering overfitting in the embedding layer caused by high-dimensional data sparsity as a critical issue. To address this, it proposes a framework called Multi-Epoch learning with Data Augmentation (MEDA), which decouples the embedding layer and the data and reinitializes embedding parameters at each epoch. The results demonstrate its effectiveness in non-incremental and incremental learning scenarios, with benefits observed in real-world applications.
weaknesses: 1,Limited Novelty of Proposed Framework: The innovation in this work appears limited to reinitializing embeddings each epoch. The paper suggests that embedding overfitting stems from initial embeddings over-memorizing data specifics across epochs; however, this can often be mitigated by reshuffling the data between epochs, an approach that isn’t fully explored here.

2,Lack of Experimental Breadth: The experimental comparisons mainly focus on one-epoch versus multi-epoch MEDA trials across different CTR models (e.g., DIN, DIEN), but they do not include other techniques for addressing embedding overfitting, such as embedding dropout, data reshuffling, or regularization. Including these would provide a more comprehensive evaluation of MEDA’s effectiveness.

3,Marginal AUC Improvement in Online Experiments: While the online experiments indicate a slight AUC improvement, there’s limited evidence of improvement in CTR, the paper’s primary focus. Since CTR is critical to the study, clearer gains in CTR would make a stronger case for MEDA, alongside secondary metrics like revenue.

4,Training Instability Due to Frequent Reinitialization: Reinitializing embeddings every epoch may cause instability, as the MLP layer must continually adapt to the changing embeddings, leading to performance fluctuations. A comparison with alternative methods like data reshuffling could offer a clearer view of MEDA’s trade-offs, particularly around stability.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: See weaknesses
soundness: 3
strengths: 1. The proposed MEDA framework is simple yet effective on the listed datasets.
2. The proposed method has proven effective with both online and offline experiments.
3. The presentation of this paper is easy-to-follow.
summary: This paper targets the single-epoch phenomenon in the CTR domain. The author identifies the overfitting as coming from the embedding layer. Specifically, a multi-epoch data augmentation method(MEDA) is proposed to study the disentangling of the dependency between embedding and the MLP layer. Both an incremental and non-incremental approach are proposed. The proposed method has proven effective with both online and offline experiments.
weaknesses: 1. The hyper-parameter in all experiments is not carefully tuned. Hence, we don't know if the effectiveness of MEDA is caused by the method itself or the weak baselines.
2. The generalizability of the one-epoch phenomenon and MEDA is over-emphasized. Not all CTR datasets witness a one-epoch phenomenon. For instance, no paper ever reported such a phenomenon on Criteo, which can be considered one of the most important datasets in CTR prediction. Based on the personal experimental experience, the author personally feels Criteo does not fit into this case.
3. The paper also lacks a theoretical analysis of what causes the one-epoch phenomenon and why MEDA can alleviate such a phenomenon. Appendix A.4 fails to make the reviewer understand these points.
4. MEDA is only tested on the MLP backbone. Other backbones such as transformer and resnet are missing in this paper.
5. The paper lacks discussion and comparison with existing model retraining techniques.
6. No code artifact is provided, undermining the reproducibility of this paper.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. The term SEL is not defined upon its first occurrence.
2. In the related work, the authors mention graph learning and state that "graph learning research has delved into fine-tuning pre-trained models for new graphs." However, there is no clear connection drawn between graph learning and CTR prediction research, particularly for this paper. The authors are suggested to explicitly state the relevance of graph learning to the work on CTR prediction, or just remove this section if it's not directly related.
3. The paper states, "Figure 3 shows that the embedding overfits on each trained data sample," but the plot does not track individual data samples. It is unclear how the authors determine that overfitting occurs on each sample. For example, the training loss may drop and then slightly increase without MEDA, and the testing loss may also fluctuate. The authors can clarify how they drew this conclusion from the data presented in Figure 3.
4. The paper claims, "the initial embedding of the second epoch precisely memorizes the information of any data sample in the first epoch." However, there is no empirical evidence provided to support this statement. Given that models typically experience information loss during training, this claim seems unlikely. The authors are suggested to provide empirical evidence supporting this claim, or to clarify if this is a hypothesis rather than a proven fact.
5. The sentence "Because, first, Figure 2(a)..." contains a syntax error.
6. On line 269, the figure reference may be incorrect. The text states, "Furthermore, our results in Figure 4(a)..." but the figure shows the cosine similarity between MLPs, not the convergence of MLPs.
7. On line 311, the details about incremental MEDA are unclear. Specifically, the algorithm "selected based on requirements such as computation/storage costs" is not described. How are these selections made according to computation and storage costs? Additionally, Figure 1(b) mentions two operations, "copy" and "select," but the "copy" operation is not described in Algorithm 2.
8. In the A/B testing section, the authors indicate that they use incremental MEDA. However, the reported numbers (test AUC increase, duration of A/B test, training time) are identical to those reported in the first Arxiv version, which only discussed non-incremental MEDA in that experiment. It is unclear whether two separate A/B tests were conducted or if the same results are being reused.
soundness: 3
strengths: The one-epoch overfitting (OEO) phenomenon is a common issue often encountered in industry and has been reported in previous work. This paper specifically investigates this issue and proposes a simple workaround to enable multi-epoch learning, thereby improving model performance. In summary, the paper has the following strengths:
+ The paper reproduces the OEO phenomenon in open benchmark datasets, validating its existence.
+ Extensive experiments and analyses are conducted to identify the main cause of OEO, which is the overfitting of embedding parameters.
+ The paper presents a simple solution to avoid OEO by randomly reinitializing embedding parameters at the beginning of each epoch, a method tailored for both incremental and non-incremental learning scenarios.
+ Both offline experiments and online A/B tests have been conducted to validate the effectiveness of the proposed method.
summary: This paper investigates the one-epoch overfitting (OEO) phenomenon in CTR prediction models and empirically identifies the overfitting of embedding parameters as the primary cause. Based on this insight, the authors propose a method of randomly reinitializing embedding parameters at the beginning of each epoch and continually learning the parameters of MLPs. This proposed method is simple yet effective, enabling multi-epoch learning.
weaknesses: I think the paper has the following major limitations that need further improvement.

1. The paper claims, 'We provide theoretical analyses of the reason,' but I did not find any theoretical analysis throughout the paper. The authors are suggested to give a theoretical analysis before the approach.
2. The authors primarily experiment with the Amazon dataset and presume the existence of one-epoch overfitting (OEO). However, according to the results reported in the benchmark paper 'Open Benchmarking for Click-Through Rate Prediction,' OEO does not always occur (some datasets exhibit OEO, while others do not; some models experience OEO, while others do not). Given these variations, the paper lacks a rigorous analysis of the conditions under which OEO occurs. Comparing datasets where OEO occurs with those where it does not could better help identify the causes, for example, by highlighting differences between the datasets.
3. While the proposed method to avoid OEO is simple, it introduces multiple groups of embedding parameters in the incremental MEDA setting. Given that the size of embedding parameters is substantial, this approach will require multiple times the memory resources. I am unsure whether the return on investment (ROI) is sufficiently high to justify this increased resource usage. The authors are suggested to include some metrics on memory usage to indicate the practicality of the method.
4. The method is simple and effective. But the authors lack a discussion whether some alternatives are considered, for example whether using a smaller learning rate for embeddings and a larger learning rate for MLPs could mitigate the OEO issue.

---

confidence: 4
contribution: 4
initial_rating: 6
presentation: 4
questions: See weakness
soundness: 4
strengths: 1. The one-epoch problem has long been an unresolved challenge, and I’m glad to see the authors making further in-depth exploration in this area—it’s truly fascinating.
2. According to my understanding, the core of Non-incremental MEDA is that the MLP part is trained normally, and the Embedding part is randomly initialized with each epoch. Simple approach, experimentally effective.
summary: The authors propose a novel Multi-Epoch learning with Data Augmentation (MEDA) framework, covering both non-incremental and incremental learning settings. The experimental results of both public and business datasets show that MEDA effectively achieves the desired effect of data augmentation and MEL can outperform the conventional SEL by a significant margin.
weaknesses: 1. The authors claim that it is easy to overfit the Embedding layer at the first epoch, so why does the collaborative filtering model not have this problem?
2. Algorithms 1 and 2 are not written clearly. For example, the $E_r$ for lines 276 and 277 look the same, but why do I have to repeat the explanation. Feel that the writing is relatively messy, and can not make people understand what the algorithm is doing at once.
3. In Figure 3, the AUC without MEDA, why is the training phase flat, but the test phase shows a zigzag shape?
4. In Figure 3, the AUC rises zigzag in the presence of MEDA. In Figure 5, why is it smooth again? Their horizontal and vertical coordinates are the same.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 1
questions: See above.
soundness: 3
strengths: 1. Provide an effective solution that addresses the multi-epochs overfitting problem.

2. Extensive experiments on public datasets and industrial scenarios across several baselines.
summary: This paper studies the multi-epochs overfitting problem in CTR prediction tasks. A solution that maintains multiple groups of embeddings is introduced to solve the problem. Experiments on five datasets are conducted to demonstrate the effectiveness.
weaknesses: 1. The key insight in Figure 2 needs more analysis. "Data 1 Emb as Fixed" also leads to overfitting, which may be caused by the new categorical ids in Data 2. Please conduct an analysis separating the performance between already-seen IDs and new IDs.

2. Limited contribution. The reason for multi-epoches overfitting has already been studied in [1]. The proposed solution, especially for incremental learning, seems just to maintain k-replications (with different but random initialization) embeddings for k epochs respectively, to avoid learning from already fitted embeddings. I suggest that the authors should provide a more in-depth analysis of how the provided solution differs from or improves upon simply maintaining k-replications of embeddings.

3. Writing issue: some concepts are not well-explained. Does the absolution position (in Sec 4.1) refer to the feature sign? What does the training algorithm (in Sec 4 Non-incremental Learning) mean? Why do multiple groups of embedding form distinct embedding spaces?

4. Reproducibility and Open-Sourcing: Details in the experiments are not clear, and the paper does not mention if the code will be made available public. Ensuring reproducibility is critical for broader validation and adoption of the proposed techniques.