SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhengyi Yang† Jiancan Wu‡* Zhicai Wang† Yancheng Yuan§* Xiang Wang†† Xiangnan He††
$^{\ddagger}$ University of Science and Technology of China
$^{\S}$ The Hong Kong Polytechnic University
{yangzhy, wangzhic}@mail.ustc.edu.cn
{wujcan,xiangwang1223,xiangnanhe}@gmail.com
yancheng.yuan@polyu.edu.hk
Sequential recommendation aims to recommend the next item that matches a user's interest, based on the sequence of items he/she interacted with before. Scrutinizing previous studies, we can summarize a common learning-to-classify paradigm — given a positive item, a recommender model performs negative sampling to add negative items and learns to classify whether the user prefers them or not, based on his/her historical interaction sequence. Although effective, we reveal two inherent limitations: (1) it may differ from human behavior in that a user could imagine an oracle item in mind and select potential items matching the oracle; and (2) the classification is limited in the candidate pool with noisy or easy supervision from negative samples, which dilutes the preference signals towards the oracle item. Yet, generating the oracle item from the historical interaction sequence is mostly unexplored. To bridge the gap, we reshape sequential recommendation as a learning-to-generate paradigm, which is achieved via a guided diffusion model, termed DreamRec. Specifically, for a sequence of historical items, it applies a Transformer encoder to create guidance representations. Noising target items explores the underlying distribution of item space; then, with the guidance of historical interactions, the denoising process generates an oracle item to recover the positive item, so as to cast off negative sampling and depict the true preference of the user directly. We evaluate the effectiveness of DreamRec through extensive experiments and comparisons with existing methods. Codes and data are open-sourced at https://github.com/YangZhengyi98/DreamRec.
Sequential recommendation has long been a fundamental and important topic in many online platforms, such as e-commerce, streaming media, and social networking [1-3]. Its core task is to recommend the next item that matches user preference, based on the sequence of items he/she interacted with before. Scrutinizing recent research on sequential recommendation [4-9], we may discern a common learning-to-classify paradigm: given a sequence of historical items and a target (truly positive) item, a recommender model first performs negative sampling to append the historical interactions with some non-interacted (possibly negative) items, and then learns to classify the positive instance from the sampled negatives. Along with this line, extensive studies are conducted on evolving the model architecture (e.g., recurrent neural networks [4], convolutional neural networks
*Jiancan Wu and Yancheng Yuan are corresponding authors.
†Xiang Wang and Xiangnan He are also affiliated with Institute of Artificial Intelligence, Institute of Dataspace, Hefei Comprehensive National Science Center.
Figure 1: Illustration of learning-to-classify and learning-to-generate paradigms of sequential recommenders w.r.t. the treatment of item embeddings. In the learning-to-classify paradigm, the negative sampling technique is typically employed to classify the positive instance from the sampled negatives (left subfigure); Without negative sampling, all items are naively viewed as positive, undermining recommendation performance (middle subfigure). In contrast, the learning-to-generate paradigm leverages observed interactions to capture the underlying data-generation distribution, enabling it to generate the oracle item beyond the candidate set (right subfigure).
[10, 11] and Transformers [5, 6]) and auxiliary tasks (e.g., causal inference [7, 12], contrastive learning [8, 13], robust optimization [9]), to enhance the classification ability and characterize the user preference better.
Clearly, this learning-to-classify paradigm seeks to predict whether an item in the candidate set (i.e., positive and sampled negative items) aligns with the user preference evidenced by the historical item sequence. Despite its success, we reveal two limitations inherent in this paradigm:
- It may simplify human behavior — a user could imagine an oracle item in mind, and then compromise to a real item that best matches the oracle. By “imagine”, we mean that the oracle item is the one that a user would ideally like to interact with next, softly absorbing and representing his/her preference from previous interactions. By “compromise”, we mean that the oracle item, conceivably, is not a concrete and discrete instance limited in the candidate set, while the real item interacted with next exemplifies the oracle. Considering the example in Figure 1, the red triangle  $\triangleleft$  is the oracle item ingratiating with the interaction sequence, while the red circle  $\bullet$  is the real item close to the oracle item and satisfying the user.
- Classification upon the candidate set is a circuitous way to characterize the user preference, to address the one-class problem posed by the availability of only positive samples [14]. Yet, as the selected negatives are confined in a small candidate set (e.g., the blue circles  $\bullet$  in Figure 1), their contrasts with the positive sample (e.g., the red circle  $\bullet$  in Figure 1) coarsen the decision boundary between what a user likes and dislikes, leaving the item space mostly unexplored. Worse still, simple negative samples may be far away from the positive item, thus failing to contribute sufficient supervision signals to the model learning [15]; whereas, the overly complex ones might be falsely negative and inject noises into the model [14, 16, 17]. Therefore, such classification signals are hard to control to depict the oracle item.
To resolve these limitations, we reshape sequential recommendation from the perspective of learning-to-generate instead. The basic idea is to describe the underlying data-generation distribution based on the historical item sequence, directly generate the oracle item that softly represents the user preference, and infer the real items most matching the oracle, as shown in Figure 1 (right subfigure). Conceptually, generating the oracle item enables us to go beyond the scope of all concrete and discrete items, and encouraging its agreement with the positive items allows us to get rid of negative sampling. Towards this end, we are setting our sights on the diffusion generative models [18-22]. The key idea of diffusion generative models is to gradually convert the data into noise, and generate samples by the parameterized denoising process. Yet, generating oracle items in sequential recommendation via diffusion models is mostly unexplored.
To bridge this gap, we propose a simple yet effective approach DreamRec, which uses the guided diffusion idea to directly generate the oracle item tailor-made for the user history. Specifically, given a sequence of historical items, the forward process of DreamRec progressively adds noise to the target item's representation and nearly leads to a complete noise, while the reverse process gradually denoises a Gaussian noise guided by history to generate the oracle item aligned with the historical interactions. Wherein, we establish the history guidance by applying a Transformer encoder on the historical item sequence, so as to make the oracle item specialize for each sequence. As a result, DreamRec enjoys the merit of directly modeling the user preference, without relying on classification and negative sampling. On three benchmark datasets, we evaluate the effectiveness of DreamRec through extensive experiments and comparisons with existing methods [4, 10, 5, 7, 12, 8].
Sequential recommendation, which captures user preference and recommends the next items based on the historical interactions, has garnered increasing attention in academia and industry [5, 4, 23]. Most existing studies approach the task under a learning-to-classify paradigm, where the decision boundary separates the positive instance from the sampled negatives. To enhance the classification ability of sequential recommenders, existing efforts can be broadly categorized into two branches. The first branch centers around leveraging complex model architectures, such as recurrent neural networks [4], convolutional neural networks [10, 11], Transformer encoders [5, 6, 24], to encode the interaction sequences and better characterize user preferences. While the second research line is centered around the inclusion of diverse auxiliary learning tasks, including causal inference [7], contrastive learning [8, 13], and robust optimization [9].
Diffusion Models have emerged as a prevailing approach for various generative tasks, including image synthesis [25, 18], text generation [22], and molecule design [26]. Their popularity stems from the ability to precisely approximate underlying data generation distribution and offer more stable training compared to other generative models like GANs [27, 28] and VAEs [29, 30]. Three primary formulations exist in the literature [31]: denoising diffusion probabilistic models (DDPM) [32], score-based generative models (SGMs) [33, 20], and stochastic differential equations (SDEs) [21, 34]. DDPM represents the forward and reverse process as two Markov chains, leveraging the Markov property to factorize the joint distribution into the product of transition kernels. SGMs introduce a sequence of intensifying Gaussian noise to perturb data, jointly estimating the score function for all noisy data distributions. Samples are generated by chaining the score functions at decreasing noise levels with score-based sampling methods such as Monte Carlo and Langevin Dynamics[35, 20]. SDEs perturb data to noise with a stochastic differential equation [21], whose forward process can be viewed as the continuous version of DDPM and SGM. Moreover, recent advances [25, 36, 22] have also enabled control over the generation process for conditional diffusion to generate specific images and text.
To our knowledge, recent studies [37-40] have explored integrating diffusion models into sequential recommendation. However, these approaches still adhere to the learning-to-classify paradigm, inevitably requiring negative sampling during training. For instance, Li et al. [37] and Du et al. [38] apply softmax cross-entropy loss [41] on the predicted logits of candidates, treating all non-target items as negative samples. While Wang et al. [39] uses binary cross-entropy loss, taking the next item as positive and randomly sampling a non-interacted item as the negative counterpart. They also incorporate contrastive loss for better classification, which requires substantial negative samples. However, diffusion model is mostly used for adding noise in the training samples for robustness, and the learning objectives are largely categorized as classification instead of generation. DiffRec by Wang et al. [40] proposes to apply diffusion on user's interaction vectors (i.e., multi-hot vectors) for collaborative recommendation, where “1” indicates a positive interaction and “0” suggests a potential negative.
In contrast, our proposed DreamRec reshapes sequential recommendation as a learning-to-generate task. Specifically, DreamRec directly generates the oracle item tailored to user behavior sequence, transcending limitations of the concrete items in the candidate set and encouraging exploration of the underlying data distribution without the need of negative sampling.
We recap the basic notions of diffusion model, as established by the pioneering DDPM framework [32]. In this paper, we define the lower-case letter in bold as a variable, whose superscript refers to the diffusion step. We keep other notations the same as DDPM [32].
The fundamental objective of a generative model parameterized by  $\theta$  is to model the underlying data-generation distribution, denoted by  $p_{\theta}(\mathbf{x}^0)$ , where  $\mathbf{x}^0$  is the target variable. DDPM, a representative formulation of diffusion model, formulates two Markov chains, leveraging the chain rule of probabilities and the Markov property, to model the underlying distribution.
In the forward (noising) process, DDPM gradually adds Gaussian noise to  $\mathbf{x}^0$  with a variance schedule  $[\beta_1,\beta_2,\dots ,\beta_T]$ :
$$
q \left(\mathbf {x} ^ {1: T} \mid \mathbf {x} ^ {0}\right) = \prod_ {t = 1} ^ {T} q \left(\mathbf {x} ^ {t} \mid \mathbf {x} ^ {t - 1}\right), \quad q \left(\mathbf {x} ^ {t} \mid \mathbf {x} ^ {t - 1}\right) = \mathcal {N} \left(\mathbf {x} ^ {t}; \sqrt {1 - \beta_ {t}} \mathbf {x} ^ {t - 1}, \beta_ {t} \mathbf {I}\right). \tag {1}
$$
Let  $\alpha_{t} = 1 - \beta_{t}$ ,  $\bar{\alpha}_{t} = \prod_{s=1}^{t} \alpha_{s}$ ,  $\epsilon \sim \mathcal{N}(0, \mathbf{I})$ , we have  $\mathbf{x}^t = \sqrt{\bar{\alpha}_t} \mathbf{x}^0 + \sqrt{1 - \bar{\alpha}_t} \epsilon$ . DDPM jointly models the target variable  $\mathbf{x}^0$  alongside a set of latent variables denoted by  $\mathbf{x}^1, \ldots, \mathbf{x}^T$  as a Markov chain with Gaussian transitions:
$$
p _ {\theta} \left(\mathbf {x} ^ {0: T}\right) = p \left(\mathbf {x} ^ {T}\right) \prod_ {t = 1} ^ {T} p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}\right), \quad p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}\right) = \mathcal {N} \left(\mathbf {x} ^ {t - 1}; \boldsymbol {\mu} _ {\theta} \left(\mathbf {x} ^ {t}, t\right), \boldsymbol {\Sigma} _ {\theta} \left(\mathbf {x} ^ {t}, t\right)\right), \tag {2}
$$
where the initial state is a Gaussian noise  $\mathbf{x}^T\sim \mathcal{N}(\mathbf{0},\mathbf{I})$ . This is called the reverse (denoising) process of DDPM.
At the core of the generation task is to optimize the underlying data generating distribution  $p_{\theta}(\mathbf{x}^{0})$ , which is performed by optimizing the variational bound of negative log-likelihood. In DDPM, this equals minimizing the KL divergence between  $q(\mathbf{x}^{0:T})$  and  $p_{\theta}(\mathbf{x}^{0:T})$ :
$$
\begin{array}{l} \mathbb {E} \left[ - \log p _ {\theta} \left(\mathbf {x} ^ {0}\right) \right] \leq D _ {K L} \left(q \left(\mathbf {x} _ {0}, \mathbf {x} _ {1}, \dots , \mathbf {x} _ {T}\right) \| p _ {\theta} \left(\mathbf {x} _ {0}, \mathbf {x} _ {1}, \dots , \mathbf {x} _ {T}\right)\right) (3) \\ = \mathbb {E} _ {q \left(\mathbf {x} _ {0}, \mathbf {x} _ {1}, \dots , \mathbf {x} _ {T}\right)} \left[ - \log p \left(\mathbf {x} _ {T}\right) - \sum_ {t = 1} ^ {T} \log \frac {p _ {\theta} \left(\mathbf {x} _ {t - 1} \mid \mathbf {x} _ {t}\right)}{q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {t - 1}\right)} \right] + C _ {1} (4) \\ = \sum_ {t = 1} ^ {T} \underbrace {D _ {K L} \left(q \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t} , \mathbf {x} ^ {0}\right) \mid p _ {\theta} \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}\right)\right)} _ {:= L _ {t - 1}} + C _ {2} (5) \\ \end{array}
$$
where  $C_1$  and  $C_2$  are constants that are independent of the model parameter  $\theta$ . Using Bayes' theorem, the posterior distribution  $q(\mathbf{x}^{t - 1}|\mathbf{x}^t,\mathbf{x}^0)$  could be solved in closed form:
$$
q \left(\mathbf {x} ^ {t - 1} \mid \mathbf {x} ^ {t}, \mathbf {x} ^ {0}\right) = \mathcal {N} \left(\mathbf {x} ^ {t - 1}; \tilde {\boldsymbol {\mu}} _ {t} \left(\mathbf {x} ^ {t}, \mathbf {x} ^ {0}\right), \tilde {\beta} _ {t} \mathbf {I}\right), \tag {6}
$$
$$
\tilde {\boldsymbol {\mu}} _ {t} \left(\mathbf {x} ^ {t}, \mathbf {x} ^ {0}\right) = \frac {\sqrt {\bar {\alpha} _ {t - 1}} \beta_ {t}}{1 - \bar {\alpha} _ {t}} \mathbf {x} ^ {0} + \frac {\sqrt {\alpha_ {t}} \left(1 - \bar {\alpha} _ {t}\right)}{1 - \bar {\alpha} _ {t}} \mathbf {x} ^ {t} \quad \text {a n d} \quad \tilde {\beta} _ {t} = \frac {1 - \bar {\alpha} _ {t - 1}}{1 - \bar {\alpha} _ {t}} \beta_ {t}. \tag {7}
$$
Further reparameterizing  $\pmb{\mu}_{\theta}(\mathbf{x}^t,t)$  as:
$$
\boldsymbol {\mu} _ {\theta} \left(\mathbf {x} ^ {t}, t\right) = \frac {1}{\sqrt {\alpha_ {t}}} \left(\mathbf {x} ^ {t} - \frac {1 - \alpha_ {t}}{\sqrt {1 - \bar {\alpha} _ {t}}} \boldsymbol {\epsilon} _ {\theta} \left(\mathbf {x} ^ {t}, t\right)\right), \tag {8}
$$
the  $t$ -th term of the training objective in Equation (5) is simplified to:
$$
L _ {t - 1} = \mathbb {E} _ {\mathbf {x} ^ {0}, \boldsymbol {\epsilon}} \left[ \frac {\beta_ {t} ^ {2}}{2 \tilde {\beta} _ {t} \alpha_ {t} (1 - \bar {\alpha} _ {t})} | | \boldsymbol {\epsilon} _ {0} - \boldsymbol {\epsilon} _ {\theta} (\sqrt {\bar {\alpha} _ {t}} \mathbf {x} ^ {0} + \sqrt {1 - \bar {\alpha} _ {t}} \boldsymbol {\epsilon}, t) | | ^ {2} \right] + C. \tag {9}
$$
Following DDPM [32],  $\Sigma_{\theta}(\mathbf{x}^t,t)$  is set to  $\tilde{\beta}_t\mathbf{I}$  to match the variance of Equation (2) and Equation (6). The model architecture of  $\epsilon_{\theta}$  depends on specific tasks, such as U-Net for image generation [32] and Transformer for text generation [22].
In this section, we elaborate on the proposed DreamRec, following the learning-to-generate paradigm with guided diffusion. We start by reformulating sequential recommendation as an oracle item generation task. Subsequently, we explain our approach to directly generate oracle item embeddings, taking inspiration from DDPM [32]. To conclude, we introduce our approach to enable personalized generation using classifier-free guidance.
4.1 Sequential Recommendation as Oracle Item Generation
Let  $\mathcal{I}$  be the set of discrete items in the dataset. A historical interaction sequence is represented as  $v_{1:n-1} = [v_1, v_2, \ldots, v_{n-1}]$ , and  $v_n$  is the subsequent consumed item of the sequence. Let  $\mathcal{D} = \{[v_{1:n-1}, v_n]_m\}_{m=1}^{|D|}$  stand for all the sequences within the training data, and  $\mathcal{D}_t = \{[v_{1:n-1}]_m\}_{m=1}^{|D_t|}$  denotes test sequences. Typically, each item  $v \in \mathcal{I}$  is initially translated into its corresponding embedding vector  $\mathbf{e}$ . Thus, the interaction sequence can be represented as  $\mathbf{e}_{1:n-1} = [\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_{n-1}]$ . The fundamental objective of sequential recommendation is to recommend the potential subsequent item that aligns with the user preference.
In this work, we propose DreamRec, a method that reshapes sequential recommendation as a learning-to-generate task. The principle behind it is that users tend to create an "oracle" item in their minds — an idealized item that they then search for in tangible form from the candidate set, when making a purchase. We assume that these oracle items are drawn from the same underlying distribution that generates observed items. In DreamRec, we model the underlying distribution as  $p_{\theta}(\mathbf{e}_n^0 | \mathbf{e}_{1:n-1})$  (i.e.,  $p_{\theta}(\mathbf{e}_n | \mathbf{e}_{1:n-1})$ ), since the generation of next item  $\mathbf{e}_n^0$  is highly related to historical interactions  $\mathbf{e}_{1:n-1}$  in sequential recommendation. If  $p_{\theta}(\mathbf{e}_n^0 | \mathbf{e}_{1:n-1})$  can be precisely learned, it becomes possible to generate the oracle item from the interaction sequence through  $p_{\theta}(\cdot | \mathbf{e}_{1:n-1})$ . In DreamRec, we learn  $p_{\theta}(\mathbf{e}_n^0 | \mathbf{e}_{1:n-1})$  by employing a guided diffusion model.
4.2 Oracle Item Generation with Guided Diffusion
After framing sequential recommendation as an oracle item generation task, we proceed to introduce the learning and generation phases of DreamRec.
4.2.1 Learning Phase of DreamRec
A straightforward application of DDPM, as outlined in Section 3, cannot sufficiently achieve the oracle item generation objective in sequential recommendation. This is primarily because the denoising process modeled in Equation (2) lacks guidance from historical interactions, resulting in non-personalized generated items. To resolve this, we propose to guide the denoising process with the corresponding historical interaction sequence. Specifically, we first encode the interaction sequence  $\mathbf{e}_{1:n - 1} = [\mathbf{e}_1,\mathbf{e}_2,\dots ,\mathbf{e}_{n - 1}]$  with a Transformer encoder:
$$
\mathbf {c} _ {n - 1} = \mathbf {T} \text {- e n c} (\mathbf {e} _ {1: n - 1}). \tag {10}
$$
This encoded interaction sequence,  $\mathbf{c}_{n - 1}$  conditions the denoising process as follows:
$$
p _ {\theta} \left(\mathbf {e} _ {n} ^ {t - 1} \mid \mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}\right) = \mathcal {N} \left(\mathbf {e} _ {n} ^ {t - 1}; \boldsymbol {\mu} _ {\theta} \left(\mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}, t\right), \boldsymbol {\Sigma} _ {\theta} \left(\mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}, t\right)\right), \tag {11}
$$
where the architecture of  $\pmb{\mu}_{\theta}(\mathbf{e}_n^t,\mathbf{c}_{n - 1},t)$  is an MLP in DreamRec depicted in Figure 2.
Similar to Equation (1), the forward process is formulated as a Markov chain of Gaussian transitions:  $q(\mathbf{e}_n^t | \mathbf{e}_n^{t-1}) = \mathcal{N}(\mathbf{e}_n^t; \sqrt{1 - \beta_t} \mathbf{e}_n^{t-1}, \beta_t \mathbf{I})$  with a variance schedule  $[\beta_1, \beta_2, \dots, \beta_T]$ . Both the forward and reverse processes of DreamRec are also illustrated in Figure 2.
Figure 2: Framework of DreamRec. To achieve personalized denoising, DreamRec encodes historical interactions  $\mathbf{e}_{1:n - 1}$  to be guidance signal  $\mathbf{c}_{n - 1}$  with a Transformer encoder T-enc, subsequently utilizing  $\mathbf{c}_{n - 1}$  to guide the reverse process.
Algorithm 1 Training phase of DreamRec
1: repeat  
2:  $\mathbf{e}_{n}^{0}, \mathbf{e}_{1:n-1} \sim \mathcal{D}$   
3:  $\mathbf{c}_{n-1} = \mathrm{T - enc}(\mathbf{e}_{1:n-1})$   
4: With probability  $p_{u}$ :  $\mathbf{c}_{n-1} = \Phi$   
5:  $t \sim \mathrm{Uniform}(\{1, \dots, T\})$   
6:  $\epsilon \sim \mathcal{N}(\mathbf{0}, \mathbf{I})$   
7:  $\mathbf{e}_{n}^{t} = \sqrt{\bar{\alpha}_{t}} \mathbf{e}_{n}^{0} + \sqrt{1 - \bar{\alpha}_{t}} \epsilon$   
8:  $\theta = \theta - \mu \nabla_{\theta} \left\| \mathbf{e}_{n}^{0} - f_{\theta}(\mathbf{e}_{n}^{t}, \mathbf{c}_{n-1}, t) \right\|^{2}$   
9: until converged
Algorithm 2 Generation phase of DreamRec
1:  $\mathbf{e}_{1:n - 1}\sim \mathcal{D}_t$  ▷ Sample and embed a data from testing set.  
2:  $\mathbf{e}_n^T\sim \mathcal{N}(\mathbf{0},\mathbf{I})$  ▷ Sample Gaussian noise.  
3:  $\mathbf{c}_{n - 1} = \mathrm{T}$  enc(e1:n-1) ▷ Encode interaction sequence.  
4: for  $t = T,\dots ,1$  do ▷ Denoise for T steps.  
5:  $\mathbf{z}\sim \mathcal{N}(\mathbf{0},\mathbf{I})$  if  $t > 1$  , else  $\mathbf{z} = \mathbf{0}$  ▷ Sample denoising variance.  
6:  $\tilde{f}_{\theta}(\mathbf{e}_{n}^{t},\mathbf{c}_{n - 1},t) = (1 + w)f_{\theta}(\mathbf{e}_{n}^{t},\mathbf{c}_{n - 1},t) - wf_{\theta}(\mathbf{e}_{n}^{t},\Phi ,t)$  ▷ Control the strength of guidance.  
7:  $\mathbf{e}_n^{t - 1} = \frac{\sqrt{\bar{\alpha}_{t - 1}\beta_t}}{1 - \bar{\alpha}_t}\tilde{f}_{\theta}(\mathbf{e}_n^t,\mathbf{c}_{n - 1},t) + \frac{\sqrt{\alpha_t(1 - \bar{\alpha}_{t - 1})}}{1 - \bar{\alpha}_t}\mathbf{e}_n^t +\sqrt{\tilde{\beta}_t}\mathbf{z}$  ▷ Denoise for one step.  
8: end for  
9: return  $\mathbf{e}_n^0$
Thereafter, DreamRec improves upon  $L_{t-1}$  of Equation (5) with guidance signal, and achieves the learning objective:
$$
L _ {t - 1} = D _ {K L} \left(q \left(\mathbf {e} _ {n} ^ {t - 1} \mid \mathbf {e} _ {n} ^ {t}, \mathbf {e} _ {n - 1} ^ {0}\right) \| p _ {\theta} \left(\mathbf {e} _ {n} ^ {t - 1} \mid \mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}\right)\right). \tag {12}
$$
We employ another reparameterization that predicts target sample  $\mathbf{e}_n^0$  rather than the added noise  $\epsilon$  as in Equation (8):
$$
\boldsymbol {\mu} _ {\theta} \left(\mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}, t\right) = \sqrt {\bar {\alpha} _ {t - 1}} f _ {\theta} \left(\mathbf {e} _ {n} ^ {t}, \mathbf {c} _ {n - 1}, t\right) + \frac {\sqrt {\alpha_ {t}} \left(1 - \bar {\alpha} _ {t - 1}\right)}{\sqrt {1 - \bar {\alpha} _ {t}}} \boldsymbol {\epsilon}, \tag {13}
$$
and Equation (12) converts to another format: