SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (A z) ] \\ = \mathbb {E} _ {z \sim \mathbb {P} (a), u \sim \mathsf {N} (0, I _ {d})} [ g ^ {*} (\alpha (t) A z + \sqrt {h (t)} A u)) ] \\ \leq \mathbb {E} _ {z \sim \mathbb {P} (a)} \left[ g ^ {*} (\alpha (t _ {0}) A z)) \right] + \sqrt {h (t _ {0})} \mathbb {E} _ {u \sim \mathsf {N} (0, I _ {d})} \left[ \| A u \| _ {2} \right] \\ \leq \mathbb {E} _ {z \sim \mathbb {P} (a)} [ g ^ {*} (A z)) ] + (1 - \alpha (t _ {0})) \mathbb {E} _ {z \sim \mathbb {P} (a)} [ \| A z \| _ {2} ] + \sqrt {h (t _ {0})} \mathbb {E} _ {u \sim \aleph (0, 1 _ {\mathrm {d}})} [ \| A u \| _ {2} ], \\ \end{array}
$$
$$
\left| \mathbb {E} _ {z \sim \mathbb {P} (a)} [ g ^ {*} (A z) ] - \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (A z) ] \right| \leq t _ {0} \cdot \mathbb {E} _ {z \sim \mathbb {P} (a)} [ \| z \| _ {2} ] + d,
$$
where we further use  $1 - \alpha(t_0) = 1 - e^{-t_0/2} \leq t_0/2$ ,  $h(t_0) \leq 1$ .
As for  $\left|\mathbb{E}_{z\sim \mathbb{P}_{t_0}(a)}[g^* (Az)] - \mathbb{E}_{z\sim (V^\top)_\#}\widehat{P}_a[g^* (Vz)]\right|$ , we have
$$
\begin{array}{l} \left| \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (A z) ] - \mathbb {E} _ {z \sim (U ^ {\top} V ^ {\top}) \#} \widehat {P} _ {a} [ g ^ {*} (V U z) ] \right| \\ = \left| \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (V U z) ] - \mathbb {E} _ {z \sim (U ^ {\top} V ^ {\top}) _ {\#} \widehat {P} _ {a}} [ g ^ {*} (V U z) ] \right| + \left| \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (A z) ] - \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (V U z) ] \right|, \\ \end{array}
$$
$$
\left| \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (A z) ] - \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (V U z) ] \right| \leq \| A - V U \| _ {F} \cdot \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ \| z \| _ {2} ],
$$
$$
\left| \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ g ^ {*} (V U z) ] - \mathbb {E} _ {z \sim (U ^ {\top} V ^ {\top}) _ {\#}} \widehat {P} _ {a} [ g ^ {*} (V U z) ] \right| \leq T V (\widehat {P} _ {a}) \cdot \| g ^ {*} \| _ {\infty}.
$$
Combining things up, we have
$$
\begin{array}{l} \mathcal {E} _ {2} \leq \| V V ^ {\top} - A A ^ {\top} \| _ {F} \cdot \mathbb {E} _ {x \sim \widehat {P} _ {a}} [ \| x \| _ {2} ] + \| A - V U \| _ {F} \cdot \mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ \| z \| _ {2} ] \\ + t _ {0} \cdot \mathbb {E} _ {z \sim \mathbb {P} (a)} [ \| z \| _ {2} ] + d + T V (\widehat {P} _ {a}) \cdot \| g ^ {*} \| _ {\infty}. \\ \end{array}
$$
Similar to parametric case, Let  $M(a) \coloneqq \mathbb{E}_{z \sim \mathbb{P}(a)}[||z||_2^2]$ , then
$$
\mathbb {E} _ {z \sim \mathbb {P} _ {t _ {0}} (a)} [ \| z \| _ {2} ^ {2} ] \leq M (a) + t _ {0} d,
$$
expect for in nonparametric case, we can not compute  $M(a)$  out as it is not Gaussian. But still, with higher-order terms in  $n_1^{-1}$  hided, we have
$$
\begin{array}{l} \mathcal {E} _ {2} = \mathcal {O} \left(T V (\widehat {P} _ {a}) \cdot \| g ^ {*} \| _ {\infty} + t _ {0} M (a)\right) \\ = \widetilde {\mathcal {O}} \left(\sqrt {\frac {\mathcal {T} (P (x , \widehat {y} = a) , P _ {x \widehat {y}} ; \bar {\mathcal {S}})}{c _ {0}}} \cdot \epsilon_ {d i f f} (n _ {1}) \cdot \| g ^ {*} \| _ {\infty} + t _ {0} M (a)\right). \\ \end{array}
$$
H Parametric Conditional Score Estimation: Proof of Lemma D.1
Proof. We first derive a decomposition of the conditional score function similar to [8]. We have
$$
\begin{array}{l} p _ {t} (x, y) = \int p _ {t} (x, y | z) p _ {z} (z) \mathrm {d} z \\ = \int p _ {t} (x | z) p (y | z) p _ {z} (z) d z \\ = C \int \exp \left(- \frac {1}{2 h (t)} \| x - \alpha (t) A z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} \left(\theta^ {\top} z - y\right) ^ {2}\right) p _ {z} (z) \mathrm {d} z \\ \stackrel {(i)} {=} C \exp \left(- \frac {1}{2 h (t)} \| \left(I _ {D} - A A ^ {\top}\right) x \| _ {2} ^ {2}\right) \\ \cdot \int \exp \left(- \frac {1}{2 h (t)} \| A ^ {\top} x - \alpha (t) z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} \left(\theta^ {\top} z - y\right) ^ {2}\right) p _ {z} (z) \mathrm {d} z, \\ \end{array}
$$
where equality  $(i)$  follows from the fact  $AA^{\top}x\perp (I_D - AA^{\top})x$  and  $C$  is the normalizing constant of Gaussian densities. Taking logarithm and then derivative with respect to  $x$  on  $p_t(x,y)$ , we obtain
$$
\begin{array}{l} \nabla_ {x} \log p _ {t} (x, y) \\ = \frac {\alpha (t)}{h (t)} \frac {A \int z \exp \left(- \frac {1}{2 h (t)} \| A ^ {\top} x - \alpha (t) z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} (\theta^ {\top} z - y) ^ {2}\right) p _ {z} (z) \mathrm {d} z}{\int \exp \left(- \frac {1}{2 h (t)} \| A ^ {\top} x - \alpha (t) z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} (\theta^ {\top} z - y) ^ {2}\right) p _ {z} (z) \mathrm {d} z} - \frac {1}{h (t)} x. \\ \end{array}
$$
Note that the first term in the right-hand side above only depends on  $A^\top x$  and  $y$ . Therefore, we can compactly write  $\nabla_x \log p_t(x,y)$  as
$$
\nabla_ {x} \log p _ {t} (x, y) = \frac {1}{h (t)} A u \left(A ^ {\top} x, y, t\right) - \frac {1}{h (t)} x, \tag {H.1}
$$
where mapping  $u$  represents
$$
\frac {\alpha (t) \int z \exp \left(- \frac {1}{2 h (t)} \| A ^ {\top} x - \alpha (t) z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} (\theta^ {\top} z - y) ^ {2}\right) p _ {z} (z) \mathrm {d} z}{\int \exp \left(- \frac {1}{2 h (t)} \| A ^ {\top} x - \alpha (t) z \| _ {2} ^ {2}\right) \exp \left(- \frac {1}{\sigma_ {y} ^ {2}} (\theta^ {\top} z - y) ^ {2}\right) p _ {z} (z) \mathrm {d} z}.
$$
We observe that (H.1) motivates our choice of the neural network architecture  $S$  in (3.8). In particular,  $\psi$  attempts to estimate  $u$  and matrix  $V$  attempts to estimate  $A$ .
In the Gaussian design case (Assumption 4.4), we instantiate  $p_z(z)$  to the Gaussian density  $(2\pi |\Sigma|)^{-d/2} \exp \left( -\frac{1}{2} z^\top \Sigma^{-1} z \right)$ . Some algebra on the Gaussian integral gives rise to
$$
\begin{array}{l} \nabla_ {x} \log p _ {t} (x, y) = \frac {\alpha (t)}{h (t)} A B _ {t} \mu_ {t} (x, y) - \frac {1}{h (t)} (I _ {D} - A A ^ {\top}) x - \frac {1}{h (t)} A A ^ {\top} x \\ = \frac {\alpha (t)}{h (t)} A B _ {t} \left(\alpha (t) A ^ {\top} x + \frac {h (t)}{\nu^ {2}} y \theta\right) - \frac {1}{h (t)} x, \tag {H.2} \\ \end{array}
$$
where we have denoted
$$
\mu_ {t} (x, y) = \alpha (t) A ^ {\top} x + \frac {h (t)}{\nu^ {2}} y \theta \quad \mathrm {a n d} \quad B _ {t} = \left(\alpha^ {2} (t) I _ {d} + \frac {h (t)}{\nu^ {2}} \theta \theta^ {\top} + h (t) \Sigma^ {- 1}\right) ^ {- 1}.
$$
Score Estimation Error Recall that we estimate the conditional score function via minimizing the denoising score matching loss in Proposition 3.1. To ease the presentation, we denote
$$
\ell (x, y; s) = \frac {1}{T - t _ {0}} \int_ {t _ {0}} ^ {T} \mathbb {E} _ {x ^ {\prime} | x} \| \nabla_ {x ^ {\prime}} \log \phi_ {t} (x ^ {\prime} | x) - s (x ^ {\prime}, y, t) \| _ {2} ^ {2} \mathrm {d} t
$$
as the loss function for a pair of clean data  $(x,y)$  and a conditional score function  $s$ . Further, we denote the population loss as
$$
\mathcal {L} (s) = \mathbb {E} _ {x, y} [ \ell (x, y; s) ],
$$
whose empirical counterpart is denoted as  $\widehat{\mathcal{L}}(s) = \frac{1}{n_1} \sum_{i=1}^{n_1} \ell(x_i, y_i; s)$ .
To bound the score estimation error, we begin with an oracle inequality. Denote  $\mathcal{L}^{\mathrm{trunc}}(s)$  as a truncated loss function defined as
$$
\mathcal {L} ^ {\operatorname {t r u n c}} (s) = \mathbb {E} [ \ell (x, y; s) \mathbf {1} \{\| x \| _ {2} \leq R, | y | \leq R \} ],
$$
where  $R > 0$  is a truncation radius chosen as  $\mathcal{O}(\sqrt{d\log d + \log K + \log\frac{n_1}{\delta}})$ . Here  $K$  is a uniform upper bound of  $s(x,y,t)\mathbb{1}\{\| x\| _2\leq R,|y|\leq R\}$  for  $s\in S$ , i.e.,  $\sup_{s\in S}\| s(x,y,t)\mathbb{1}\{\| x\| _2\leq R,|y|\leq R\} \| _2\leq K$ . To this end, we have
$$
\begin{array}{l} \mathcal {L} (\widehat {s}) = \mathcal {L} (\widehat {s}) - \widehat {\mathcal {L}} (\widehat {s}) + \widehat {\mathcal {L}} (\widehat {s}) \\ = \mathcal {L} (\widehat {s}) - \widehat {\mathcal {L}} (\widehat {s}) + \inf  _ {s \in \mathcal {S}} \widehat {\mathcal {L}} (s) \\ \stackrel {(i)} {=} \mathcal {L} (\widehat {s}) - \widehat {\mathcal {L}} (\widehat {s}) \\ \leq \mathcal {L} (\widehat {s}) - \mathcal {L} ^ {\mathrm {t r u n c}} (\widehat {s}) + \mathcal {L} ^ {\mathrm {t r u n c}} (\widehat {s}) - \widehat {\mathcal {L}} ^ {\mathrm {t r u n c}} (\widehat {s}) \\ \leq \underbrace {\sup  _ {s} \mathcal {L} ^ {\mathrm {t r u n c}} (s) - \widehat {\mathcal {L}} ^ {\mathrm {t r u n c}} (s)} _ {(A)} + \underbrace {\sup  _ {s} \mathcal {L} (s) - \mathcal {L} ^ {\mathrm {t r u n c}} (s)} _ {(B)}, \\ \end{array}
$$
where equality  $(i)$  holds since  $S$  contains the ground truth score function. We bound term  $(A)$  by a PAC-learning concentration argument. Using the same argument in [8, Theorem 2, term  $(A)]$ , we have
$$
\sup  _ {s \in \mathcal {S}} \ell^ {\mathrm {t r u n c}} (x, y; s) = \mathcal {O} \left(\frac {1}{t _ {0} (T - t _ {0})} (K ^ {2} + R ^ {2})\right).
$$
Applying the standard metric entropy and symmetrization technique, we can show
$$
(A) = \mathcal {O} \left(\widehat {\Re} (\mathcal {S}) + \left(\frac {K ^ {2} + R ^ {2}}{t _ {0} (T - t _ {0})}\right) \sqrt {\frac {\log \frac {2}{\delta}}{2 n _ {1}}}\right),
$$
where  $\widehat{\mathfrak{N}}$  is the empirical Rademacher complexity of  $S$ . Unfamiliar readers can refer to Theorem 3.3 in "Foundations of Machine Learning", second edition for details. The remaining step is to bound the Rademacher complexity by Dudley's entropy integral. Indeed, we have
$$
\widehat {\mathfrak {R}} (\mathcal {S}) \leq \inf  _ {\epsilon} \frac {4 \epsilon}{\sqrt {n _ {1}}} + \frac {1 2}{n _ {1}} \int_ {\epsilon} ^ {K ^ {2} \sqrt {n _ {1}}} \sqrt {\mathcal {N} (\mathcal {S} , \epsilon , \| \cdot \| _ {2})} d \epsilon .
$$
We emphasize that the log covering number considers  $x, y$  in the truncated region. Taking  $\epsilon = \frac{1}{n_1}$  gives rise to
$$
(A) = \mathcal {O} \left(\left(\frac {K ^ {2} + R ^ {2}}{t _ {0} (T - t _ {0})}\right) \sqrt {\frac {\mathcal {N} (\mathcal {S} , 1 / n _ {1}) \log \frac {1}{\delta}}{n _ {1}}}\right).
$$
Here  $K$  is instance dependent and majorly depends on  $d$ . In the Gaussian design case, we can verify that  $K$  is  $\mathcal{O}(\sqrt{d})$ . To this end, we deduce  $(A) = \widetilde{\mathcal{O}}\left(\frac{1}{t_0}\sqrt{d^2\frac{\mathcal{N}(S,1/n_1)\log\frac{1}{\delta}}{n_1}}\right)$ . In practice,  $d$  is often much smaller than  $D$  (see for example [37], where ImageNet has intrinsic dimension no more than 43 in contrast to image resolution of  $224\times 224\times 3$ ). In this way, we can upper bound  $d^2$  by  $D$ , yet  $d^2$  is often a tighter upper bound.
For term  $(B)$ , we invoke the same upper bound in [8, Theorem 2, term  $(B)]$  to obtain
$$
(B) = \mathcal {O} \left(\frac {1}{n _ {1} t _ {0} (T - t _ {0})}\right),
$$
which is negligible compared to  $(A)$ . Therefore, summing up  $(A)$  and  $(B)$ , we deduce
$$
\epsilon_ {d i f f} ^ {2} = \mathcal {O} \left(\frac {1}{t _ {0}} \sqrt {\frac {\mathcal {N} (\mathcal {S} , 1 / n _ {1}) (d ^ {2} \vee D) \log \frac {1}{\delta}}{n _ {1}}}\right).
$$
Gaussian Design We only need to find the covering number under the Gaussian design case. Using (H.2), we can construct a covering from coverings on matrices  $V$  and  $\Sigma^{-1}$ . Suppose  $V_{1}, V_{2}$  are two matrices with  $\| V_{1} - V_{2} \|_{2} \leq \eta_{V}$  for some  $\eta > 0$ . Meanwhile, let  $\Sigma_{1}^{-1}, \Sigma_{2}^{-1}$  be two covariance matrices with  $\| \Sigma_{1}^{-1} - \Sigma_{2}^{-1} \|_{2} \leq \eta_{\Sigma}$ . Then we bound
$$
\begin{array}{l} \sup  _ {\| x \| _ {2} \leq R, | y | \leq R} \| s _ {V _ {1}, \Sigma_ {1} ^ {- 1}} (s, y, t) - s _ {V _ {2}, \Sigma_ {2} ^ {- 1}} (x, y, t) \| _ {2} \\ \leq \frac {1}{h (t)} \sup  _ {\| x \| _ {2} \leq R, | y | \leq R} \left[ \left\| V _ {1} \psi_ {\Sigma_ {1} ^ {- 1}} \left(V _ {1} ^ {\top} x, y, t\right) - V _ {1} \psi_ {\Sigma_ {1} ^ {- 1}} \left(V _ {2} ^ {\top} x, y, t\right) \right\| _ {2} \right. \\ + \underbrace {\left\| V _ {1} \psi_ {\Sigma_ {1} ^ {- 1}} \left(V _ {2} ^ {\top} x , y , t\right) - V _ {1} \psi_ {\Sigma_ {2} ^ {- 1}} \left(V _ {2} ^ {\top} x , y , t\right) \right\| _ {2}} _ {(\spadesuit)} + \left\| V _ {1} \psi_ {\Sigma_ {2} ^ {- 1}} \left(V _ {2} ^ {\top} x , y , t\right) - V _ {2} \psi_ {\Sigma_ {2} ^ {- 1}} \left(V _ {2} ^ {\top} x , y , t\right) \right\| _ {2} \Big ] \\ \leq \frac {1}{h (t)} \left(2 R \eta_ {V} + 2 \nu^ {- 2} R \eta_ {\Sigma}\right), \\ \end{array}
$$
where for bounding  $(\spadesuit)$ , we invoke the identity  $\| (I + A)^{-1} - (I + B)^{-1}\| _2\leq \| B - A\| _2$ . Further taking supremum over  $t\in [t_0,T]$  leads to
$$
\sup  _ {\| x \| _ {2} \leq R, | y | \leq R} \| s _ {V _ {1}, \Sigma_ {1} ^ {- 1}} (s, y, t) - s _ {V _ {2}, \Sigma_ {2} ^ {- 1}} (x, y, t) \| _ {2} \leq \frac {1}{t _ {0}} \left(2 R \eta_ {V} + 2 \nu^ {- 2} R \eta_ {\Sigma}\right)
$$
for any  $t \in [t_0, T]$ . Therefore, the inequality above suggests that coverings on  $V$  and  $\Sigma^{-1}$  form a covering on  $S$ . The covering numbers of  $V$  and  $\Sigma^{-1}$  can be directly obtained by a volume ratio argument; we have
$$
\mathcal {N} (V, \eta_ {V}, \| \cdot \| _ {2}) \leq D d \log \left(1 + \frac {2 \sqrt {d}}{\eta_ {V}}\right) \quad \text {a n d} \quad \mathcal {N} (\Sigma^ {- 1}, \eta_ {\Sigma}, \| \cdot \| _ {2}) \leq d ^ {2} \log \left(1 + \frac {2 \sqrt {d}}{\lambda_ {\min} \eta_ {\Sigma}}\right).
$$
Thus, the log covering number of  $S$  is
$$
\begin{array}{l} \mathcal {N} (\mathcal {S}, \eta , \| \cdot \| _ {2}) = \mathcal {N} (V, t _ {0} \eta_ {V} / 2 R, \| \cdot \| _ {2}) + \mathcal {N} (\Sigma^ {- 1}, t _ {0} \nu^ {2} \eta_ {\Sigma} / 2 R, \| \cdot \| _ {2}) \\ \leq \left(D d + d ^ {2}\right) \log \left(1 + \frac {d D}{t _ {0} \lambda_ {\min} \eta}\right), \\ \end{array}
$$
where we have plugged  $\nu^2 = 1 / D$  into the last inequality. Setting  $\eta = 1 / n_1$  and substituting into  $\epsilon_{diff}^2$  yield the desired result.
We remark that the analysis here does not try to optimize the error bounds, but aims to provide a provable guarantee for conditional score estimation using finite samples. We foresee that sharper analysis via Bernstein-type concentration may result in a better dependence on  $n_1$ . Nonetheless, the optimal dependence should not beat a  $1 / n_1$ -rate.
I.1 Simulation
We generate the latent sample  $z$  from standard normal distribution  $z \sim \mathsf{N}(0, \mathsf{I}_{\mathrm{d}})$  and set  $x = A z$  for a randomly generated orthonormal matrix  $A \in \mathbb{R}^{D \times d}$ . The dimensions are set to be  $d = 16$ ,  $D = 64$ . The reward function is set to be  $f(x) = (\theta^{\star})^{\top} x_{\parallel} + 5 \| x_{\perp} \|_2^2$ , where  $\theta^{\star}$  is defined by  $A \beta^{\star}$ . We generate  $\beta^{\star}$  by uniformly sampling from the unit sphere.
When estimating  $\widehat{\theta}$ , we set  $\lambda = 1.0$ . The score matching network is based on the UNet implementation from https://github.com/lucidrains/denoising-diffusion-pytorch, where we modified the class embedding so it accepts continuous input. The predictor is trained using 8192 samples and the score function is trained using 65536 samples. When training the score function, we choose Adam as the optimizer with learning rate  $8 \times 10^{-5}$ . We train the score function for 10 epochs, each epoch doing a full iteration over the whole training dataset with batch size 32.
For evaluation, the statistics is computed using 2048 samples generated from the diffusion model. The curve in the figures is computed by averaging over 5 runs.
I.2 Directed Text-to-Image Generation
Samples of high rewards and low rewards from the ground-truth reward model. In Section 5.2, the ground-truth reward model is built by replacing the final prediction layer of the ImageNet pretrained ResNet-18 model with a randomly initialized linear layer of scalar outputs. To investigate the meaning of this randomly-generated reward model, we generate images using Stable Diffusion and filter out images with rewards  $\geq 0.4$  (positive samples) and rewards  $\leq -0.4$  (negative samples) and pick two typical images for each; see Figure 7. We note that in real-world use cases, the ground-truth rewards are often measured and annotated by human labors according to the demands.
Training Details. In our implementation, as the Stable Diffusion model operates on the latent space of its VAE, we build a 3-layer ConvNet with residual connections and batch normalizations on top of the VAE latent space. We train the network using Adam optimizer with learning rate 0.001 for 100 epochs.
I.3 Decision-Diffuser [1]
We replicate the results in Decision Diffuser [1] under (Med-Expert, Hopper) setting. In Decision Diffuser, the RL trajectory and the final reward are jointly modeled with a conditioned diffusion model. The policy is given by first performing reward-directed conditional diffusion to sample a
(a) A positive sample
(b) A positive sample
(c) A negative sample
Figure 7: Random samples with high rewards and low rewards.
(d) A negative sample
trajectory of high reward and then extracting action sequences from the trajectory using a trained inverse dynamics model. We plot the mean and standard deviation of the total returns (averaged across 10 independent episodes) v.s. the target rewards in Figure 8. The theoretical maximum of the reward is 400. Therefore, trajectories with rewards greater than 400 are never seen during training. We observe that when we increase the target reward beyond 320, the actual total reward decreases. According to our theory, as we increase the reward guidance signal, the condition effect becomes stronger but the distribution-shift effect also becomes stronger.
Figure 8: Total Reward v.s. Target Reward for Decision Diffuser (Med-Expert, Hopper) setting.