SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \int_ {s} ^ {t} \Phi^ {3} (t, \tau) B ^ {3} (\tau) \mathrm {d} \tau = \sigma_ {t} \alpha_ {t} \int_ {s} ^ {t} \frac {- 1}{\alpha_ {\tau} \sigma_ {\tau}} \alpha_ {\tau} \frac {\dot {\sigma} _ {\tau}}{\sigma_ {\tau}} \mathrm {d} \tau = \sigma_ {t} \alpha_ {t} \int_ {s} ^ {t} \frac {- \dot {\sigma} _ {\tau}}{\sigma_ {\tau} ^ {2}} \mathrm {d} \tau \\ = \sigma_ {t} \alpha_ {t} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {\lambda} d \lambda = - \alpha_ {t} (e ^ {- h} - 1). \\ \end{array}
$$
$$
\int_ {s} ^ {t} \Phi^ {2} (t, \tau) B ^ {2} (\tau) \mathrm {d} \tau = \alpha_ {t} \int_ {s} ^ {t} \frac {2}{\alpha_ {\tau}} \alpha_ {\tau} \dot {\sigma} _ {\tau} \mathrm {d} \tau = - 2 \bar {\sigma} _ {t} (e ^ {h} - 1),
$$
and finally, as already shown in Propositions 3.1 and 4.2:
$$
\begin{array}{l} \int_ {s} ^ {t} \Phi^ {1} (t, \tau) B ^ {1} (\tau) \mathrm {d} \tau = \sigma_ {t} ^ {2} \alpha_ {t} \int_ {s} ^ {t} \frac {- 2 \dot {\sigma} _ {\tau}}{\sigma_ {\tau} ^ {3}} \mathrm {d} \tau = \sigma_ {t} ^ {2} \alpha_ {t} \int_ {s} ^ {t} - 2 \frac {\mathrm {d} \lambda_ {\tau}}{\mathrm {d} \tau} e ^ {2 \lambda_ {\tau}} \mathrm {d} \tau \\ = \sigma_ {t} ^ {2} \alpha_ {t} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {2 \lambda} \mathrm {d} \lambda = - \alpha_ {t} (e ^ {- 2 h} - 1). \\ \end{array}
$$
Now, since  $g^2(t) = 2\alpha_t^2 \dot{\sigma}_t \sigma_t$ , the stochastic integrals  $\int_s^t \Phi^i(t, \tau) g(\tau) \, \mathrm{d}\bar{\omega}_{\tau}$ , for  $i = 1, 2$ , have zero mean and variances:
$$
\int_ {s} ^ {t} (\Phi^ {1} (t, \tau)) ^ {2} g ^ {2} (\tau) \mathrm {d} \tau = \sigma_ {t} ^ {4} \alpha_ {t} ^ {2} \int_ {s} ^ {t} \frac {1}{\sigma_ {\tau} ^ {4} \alpha_ {\tau} ^ {2}} g ^ {2} (\tau) \mathrm {d} \tau = \bar {\sigma} _ {t} ^ {2} (1 - e ^ {- 2 h})
$$
$$
\int_ {s} ^ {t} (\Phi^ {2} (t, \tau)) ^ {2} g ^ {2} (\tau) \mathrm {d} \tau = \alpha_ {t} ^ {2} \int_ {s} ^ {t} 2 \dot {\sigma} _ {\tau} \sigma_ {\tau} \mathrm {d} \tau = - \bar {\sigma} _ {t} ^ {2} (e ^ {2 h} - 1).
$$
We deduce from this the SEEDS-1 schemes in all four modes as given by iterates:
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t} ^ {2} \alpha_ {t}}{\sigma_ {s} ^ {2} \alpha_ {s}} \widetilde {\mathbf {x}} _ {s} - \alpha_ {t} (e ^ {- 2 h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) + \bar {\sigma} _ {t} \sqrt {1 - e ^ {- 2 h}} \epsilon \quad \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I} _ {d}) \qquad (2 8)
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\alpha_ {t}}{\alpha_ {s}} \widetilde {\mathbf {x}} _ {s} - 2 \bar {\sigma} _ {t} \left(e ^ {h} - 1\right) \boldsymbol {\epsilon} _ {\theta} \left(\widetilde {\mathbf {x}} _ {s}, s\right) - \bar {\sigma} _ {t} \sqrt {e ^ {2 h} - 1} \epsilon \quad \epsilon \sim \mathcal {N} (\mathbf {0}, \mathbf {I} _ {d}) \tag {29}
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\bar {\sigma} _ {t}}{\bar {\sigma} _ {s}} \widetilde {\mathbf {x}} _ {s} - \alpha_ {t} \left(e ^ {- h} - 1\right) D _ {\theta} \left(\widetilde {\mathbf {x}} _ {s}, s\right) \tag {30}
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\alpha_ {t}}{\alpha_ {s}} \widetilde {\mathbf {x}} _ {s} - \bar {\sigma} _ {t} \left(e ^ {h} - 1\right) \epsilon_ {\theta} \left(\widetilde {\mathbf {x}} _ {s}, s\right). \tag {31}
$$
Notice that the iterates (30) and (31) are exactly the iterates of the first stage solvers in [22] and [23] with  $F_{\theta}(\mathbf{x}_t; (M - 1)t) = \epsilon_{\theta}(\mathbf{x}_t, t)$ . The iterates (29) coincide with the SEEDS-1 method presented in (14) and (28) consist on our SEEDS-1 method in the data prediction mode, which we will use in the following section.
B.2.5 Proof of Proposition 4.5
We will write in bold the statement to be proven.
If we set  $g = 0$  in (15), the resulting SEEDS solvers do not yield DPM-Solver. Indeed, if we set  $g = 0$  in (15), then the method (29) does not contain a noise contribution and we readily see that it cannot be equal to (31). As the latter has been shown to be DPM-Solver-1, the conclusion follows.
If we parameterize (15) in terms of the data prediction model  $D_{\theta}$ , the resulting SEEDS solvers are not equivalent to their noise prediction counterparts defined in Alg. 1 to 4.
One can check that the SEEDS solver are not the same between the noise and data prediction modes by simply noticing that the noise contributions in (28) and (29) do not equate.
The gDDIM solver [41] Theorem 1, for  $\ell = 1$ , is equal to SEEDS-1 in the data prediction mode.
As shown in (29), our proposed method SEEDS-1 in the data prediction mode for the VP case has iterates of the form
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t} ^ {2} \alpha_ {t}}{\sigma_ {s} ^ {2} \alpha_ {s}} \widetilde {\mathbf {x}} _ {s} - \alpha_ {t} \left(e ^ {- 2 h} - 1\right) D _ {\theta} \left(\widetilde {\mathbf {x}} _ {s}, s\right) + \bar {\sigma} _ {t} \sqrt {1 - e ^ {- 2 h}} \epsilon , \tag {32}
$$
where  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I}_d)$ ,  $\bar{\sigma}_{t} = \alpha_{t}\sigma_{t}$  and  $h = \log \frac{\sigma_s}{\sigma_t}$ . As our notation and that of [41, Theorem 1] overlap, we will use blue color when referring to their notation.
On the one hand, gDDIM constructs iterates over a representation of the exact solution of the following family of neural differential equations:
$$
\mathrm {d} \mathbf {u} _ {t} = \left[ f (t) \mathbf {u} _ {t} + \frac {1 + \lambda^ {2}}{2} \frac {g ^ {2} (t)}{\sqrt {1 - \alpha_ {t}}} \epsilon_ {\theta} (\mathbf {u} _ {t}; t) \right] \mathrm {d} t + \lambda g (t) \mathrm {d} \bar {\omega} _ {t}, \tag {33}
$$
where  $\alpha_{t}$  decreases from  $\alpha_0 = 1$  to  $\alpha_{T} = 0$ , and with coefficients
$$
f (t) := \frac {1}{2} \frac {\mathrm {d} \log \alpha_ {t}}{\mathrm {d} t}, \quad g (t) = \sqrt {- \frac {\mathrm {d} \log \alpha_ {t}}{\mathrm {d} t}}.
$$
In particular, they choose an approximation, for  $\tau \in [t - \Delta t, t]$ , given by
$$
\pmb {s} _ {\theta} (\mathbf {u}, \tau) = \frac {\epsilon_ {\theta} (\mathbf {u} _ {\tau} , \tau)}{\sqrt {1 - \alpha_ {t}}} \approx \frac {1 - \alpha_ {t}}{1 - \alpha_ {\tau}} \sqrt {\frac {\alpha_ {\tau}}{\alpha_ {t}}} \pmb {s} _ {\theta} (\mathbf {u} (t), t) - \frac {1}{1 - \alpha_ {\tau}} (\mathbf {u} - \sqrt {\frac {\alpha_ {\tau}}{\alpha_ {t}}} \mathbf {u} (t)).
$$
The gDDIM iterates, for  $\lambda = 1 = \ell$ , are then written as follows:
$$
\mathbf {u} (t - \Delta t) = \sqrt {\frac {\alpha_ {t - \Delta t}}{\alpha_ {t}}} \mathbf {u} (t) + \left[ - \sqrt {\frac {\alpha_ {t - \Delta t}}{\alpha_ {t}}} \sqrt {1 - \alpha_ {t}} + \sqrt {1 - \alpha_ {t - \Delta t} - \sigma_ {t} ^ {2}} \right] \epsilon_ {\theta} (\mathbf {u} (t), t) + \sigma_ {t} \epsilon , \tag {34}
$$
where  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I}_d)$  and
$$
\sigma_ {t} ^ {2} = \left(1 - \alpha_ {t - \Delta t}\right) \left[ 1 - \left(\frac {1 - \alpha_ {t - \Delta t}}{1 - \alpha_ {t}}\right) \left(\frac {\alpha_ {t}}{\alpha_ {t - \Delta t}}\right) \right]. \tag {35}
$$
Now set  $(s,t)\gets (t,t - \Delta t)$ . Then
$$
\sigma_ {s} ^ {2} = (1 - \alpha_ {t}) \left[ 1 - \left(\frac {1 - \alpha_ {t}}{1 - \alpha_ {s}}\right) \left(\frac {\alpha_ {s}}{\alpha_ {t}}\right) \right],
$$
$$
{\mathbf {u} (t)} = {\sqrt {\frac {\alpha_ {t}}{\alpha_ {s}}} \mathbf {u} (s) + \left[ \sqrt {1 - \alpha_ {t} - \sigma_ {s} ^ {2}} - \sqrt {\frac {\alpha_ {t}}{\alpha_ {s}}} \sqrt {1 - \alpha_ {s}} \right] \epsilon_ {\theta} (\mathbf {u} (s), s) + \sigma_ {s} \epsilon .}
$$
Next, we identify  $\alpha_{t} = \sqrt{\alpha_{t}}$ , and  $\bar{\sigma}_{t} = \sqrt{1 - \alpha_{t}}$ . Then the variance of the noise in (35) is
$$
\begin{array}{l} \sigma_ {s} ^ {2} = \bar {\sigma} _ {t} ^ {2} \left[ 1 - \left(\frac {\bar {\sigma} _ {t}}{\bar {\sigma} _ {s}}\right) ^ {2} \left(\frac {\alpha_ {s}}{\alpha_ {t}}\right) ^ {2} \right] = \bar {\sigma} _ {t} ^ {2} \left[ 1 - \left(\frac {\alpha_ {t} \sigma_ {t}}{\alpha_ {s} \sigma_ {s}}\right) ^ {2} \left(\frac {\alpha_ {s}}{\alpha_ {t}}\right) ^ {2} \right] \\ { = } { \bar { \sigma } _ { t } ^ { 2 } \left[ 1 - \left( \frac { \sigma _ { t } } { \sigma _ { s } } \right) ^ { 2 } \right] } \\ = \bar {\sigma} _ {t} ^ {2} (1 - e ^ {- 2 h}). \\ \end{array}
$$
Hence, by denoting  $\widetilde{\mathbf{x}}_t = \mathbf{u}(t)$ , the gDDIM iterate (34) reads, for  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I}_d)$ :
$$
\begin{array}{l} \widetilde {\mathbf {x}} _ {t} = \frac {\alpha_ {t}}{\alpha_ {s}} \widetilde {\mathbf {x}} _ {s} + \left[ \sqrt {\bar {\sigma} _ {t} ^ {2} - \bar {\sigma} _ {t} ^ {2} (1 - e ^ {- 2 h})} - \frac {\alpha_ {t}}{\alpha_ {s}} \bar {\sigma} _ {s} \right] \epsilon_ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) + \sqrt {\bar {\sigma} _ {t} ^ {2} (1 - e ^ {- 2 h})} \epsilon_ {\phi} (\widetilde {\mathbf {x}} _ {s}, s), \\ { = } { \frac { \alpha _ { t } } { \alpha _ { s } } \widetilde { \mathbf { x } } _ { s } + \left[ \bar { \sigma } _ { t } \sqrt { e ^ { - 2 h } } - \alpha _ { t } \sigma _ { s } \right] \epsilon _ { \theta } ( \widetilde { \mathbf { x } } _ { s } , s ) + \bar { \sigma } _ { t } \sqrt { 1 - e ^ { - 2 h } } \epsilon _ { s } . } \\ = \frac {\alpha_ {t}}{\alpha_ {s}} \widetilde {\mathbf {x}} _ {s} + \bar {\sigma} _ {t} \left[ \frac {\sigma_ {t}}{\sigma_ {s}} - \frac {\sigma_ {s}}{\sigma_ {t}} \right] \epsilon_ {\theta} \left(\widetilde {\mathbf {x}} _ {s}, s\right) + \bar {\sigma} _ {t} \sqrt {1 - e ^ {- 2 h}} \epsilon . \tag {36} \\ \end{array}
$$
On the other hand, the data and noise prediction models in our case are related by the following equation:
$$
D _ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) = c _ {1} (s) \frac {\widetilde {\mathbf {x}} _ {s}}{\alpha_ {s}} + c _ {2} (s) \boldsymbol {\epsilon} _ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) = \frac {\widetilde {\mathbf {x}} _ {s}}{\alpha_ {s}} - \sigma_ {s} \boldsymbol {\epsilon} _ {\theta} (\widetilde {\mathbf {x}} _ {s}, s).
$$
As such, and as  $h = \lambda_t - \lambda_s = \log \frac{\sigma_s}{\sigma_t}$ , one can rewrite (32) in terms of  $\epsilon_{\theta}$  as follows:
$$
\begin{array}{l} \widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t} ^ {2} \alpha_ {t}}{\sigma_ {s} ^ {2} \alpha_ {s}} \widetilde {\mathbf {x}} _ {s} - \alpha_ {t} (e ^ {- 2 h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) + \bar {\sigma} _ {t} \sqrt {1 - e ^ {- 2 h}} \epsilon \\ { = } { \frac { \sigma _ { t } ^ { 2 } \alpha _ { t } } { \sigma _ { s } ^ { 2 } \alpha _ { s } } \widetilde { \mathbf { x } } _ { s } - \alpha _ { t } \left( \frac { \sigma _ { t } ^ { 2 } } { \sigma _ { s } ^ { 2 } } - 1 \right) \left[ \frac { \widetilde { \mathbf { x } } _ { s } } { \alpha _ { s } } - \sigma _ { s } \pmb { \epsilon } _ { \theta } ( \widetilde { \mathbf { x } } _ { s } , s ) \right] + \bar { \sigma } _ { t } \sqrt { 1 - e ^ { - 2 h } } \epsilon _ { s } . } \\ { = } { \left[ \frac { \sigma _ { t } ^ { 2 } \alpha _ { t } } { \sigma _ { s } ^ { 2 } \alpha _ { s } } - \frac { \alpha _ { t } } { \alpha _ { s } } \left( \frac { \sigma _ { t } ^ { 2 } } { \sigma _ { s } ^ { 2 } } - 1 \right) \right] \widetilde { \mathbf { x } } _ { s } + \alpha _ { t } \left( \frac { \sigma _ { t } ^ { 2 } } { \sigma _ { s } ^ { 2 } } - 1 \right) \sigma _ { s } \boldsymbol { \epsilon } _ { \theta } ( \widetilde { \mathbf { x } } _ { s } , s ) + \bar { \sigma } _ { t } \sqrt { 1 - e ^ { - 2 h } } \boldsymbol { \epsilon } _ { s } . } \\ { = } { \frac { \alpha _ { t } } { \alpha _ { s } } \widetilde { \mathbf { x } } _ { s } + \bar { \sigma } _ { t } \left( \frac { \sigma _ { t } ^ { 2 } } { \sigma _ { s } ^ { 2 } } - 1 \right) \frac { \sigma _ { s } } { \sigma _ { t } } \boldsymbol { \epsilon } _ { \theta } ( \widetilde { \mathbf { x } } _ { s } , s ) + \bar { \sigma } _ { t } \sqrt { 1 - e ^ { - 2 h } } \boldsymbol { \epsilon } _ { s } , } \\ { = } { \frac { \alpha _ { t } } { \alpha _ { s } } \widetilde { \mathbf { x } } _ { s } + \bar { \sigma } _ { t } \left( \frac { \sigma _ { t } } { \sigma _ { s } } - \frac { \sigma _ { s } } { \sigma _ { t } } \right) \epsilon _ { \theta } ( \widetilde { \mathbf { x } } _ { s } , s ) + \bar { \sigma } _ { t } \sqrt { 1 - e ^ { - 2 h } } \epsilon , } \\ \end{array}
$$
which coincides with the gDDIM iterate in Equation (36).
B.2.6 The VE, DDIM and iDDPM cases
Following [16, Eq. 217], here  $\alpha_{t} = 1$  and  $c_{1}(t) = 1$  and so the only possibilities incurring into semi-linear differential equations are
$$
A ^ {1} (t) = 2 \frac {\dot {\sigma} _ {t}}{\sigma_ {t}} \quad B ^ {1} (t) = - 2 \frac {\dot {\sigma} _ {t}}{\sigma_ {t}} \quad (\mathrm {D P S D E})
$$
$$
A ^ {3} (t) = \frac {\dot {\sigma} _ {t}}{\sigma_ {t}} \quad B ^ {3} (t) = - \frac {\dot {\sigma} _ {t}}{\sigma_ {t}}. \quad (\mathrm {D P P F O})
$$
We obtain, again with the choice  $\lambda_t = -\log(\sigma_t)$  and setting  $h = \lambda_t - \lambda_s$ , the following:
$$
\Phi^ {1} (t, s) = \frac {\sigma_ {t} ^ {2}}{\sigma_ {s} ^ {2}}, \qquad \Phi^ {3} (t, s) = \frac {\sigma_ {t}}{\sigma_ {s}},
$$
$$
\int_ {s} ^ {t} \Phi^ {1} (t, \tau) B ^ {1} (\tau) \mathrm {d} \tau = - \sigma_ {t} ^ {2} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {2 \lambda} \mathrm {d} \lambda = \frac {1}{2} (e ^ {- 2 h} - 1)
$$
$$
\int_ {s} ^ {t} \Phi^ {3} (t, \tau) B ^ {3} (\tau) \mathrm {d} \tau = - \sigma_ {t} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {\lambda} \mathrm {d} \lambda = e ^ {- h} - 1.
$$
$$
\int_ {s} ^ {t} (\Phi^ {1} (t, \tau)) ^ {2} g ^ {2} (\tau) \mathrm {d} \tau = \sigma_ {t} ^ {4} \int_ {s} ^ {t} \frac {1}{\sigma_ {\tau} ^ {4}} \sigma_ {\tau} ^ {2} 2 \frac {\dot {\sigma} _ {\tau}}{\sigma_ {\tau}} \mathrm {d} \tau = \sigma_ {t} ^ {4} \int_ {s} ^ {t} \frac {2 \dot {\sigma} _ {\tau}}{\sigma_ {\tau} ^ {3}} \mathrm {d} \tau = \sigma_ {t} ^ {2} (e ^ {- 2 h} - 1).
$$
We readily see that these cases are identical to the VP case with  $\alpha_{t} = 1$ . In particular, the obtained SEEDS-1 iterates are
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t} ^ {2}}{\sigma_ {s} ^ {2}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- 2 h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) + \sigma_ {t} \sqrt {1 - e ^ {- 2 h}} \epsilon .
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t}}{\sigma_ {s}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}).
$$
Now denote  $s_1 = t_\lambda (\lambda_s + rh)$ , for  $0 < r \leqslant 1$ , where  $t_\lambda = e^{-\lambda}$ . There are two families of single-step one-parameter two-stage exponential ODE schemes:
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t}}{\sigma_ {s}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- h} - 1) \left[ \left(1 - \frac {1}{2 r}\right) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) + \frac {1}{2 r} D _ {\theta} (\widetilde {\mathbf {x}} _ {1}; \sigma_ {s _ {1}}) \right]
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t}}{\sigma_ {s}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) + \frac {1}{r} \left(\frac {e ^ {- h} - 1}{h} + 1\right) [ D _ {\theta} (\widetilde {\mathbf {x}} _ {1}; \sigma_ {s _ {1}}) - D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) ],
$$
with same supporting value
$$
\widetilde {\mathbf {x}} _ {1} = \frac {\sigma_ {s _ {1}}}{\sigma_ {s}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- r h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}).
$$
In the same vein we define a single-step two stage exponential SDE scheme:
$$
\begin{array}{l} \widetilde {\mathbf {x}} _ {1} = \frac {\sigma_ {s _ {1}} ^ {2}}{\sigma_ {s} ^ {2}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- 2 r h} - 1) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) + \sigma_ {s _ {1}} \sqrt {e ^ {- 2 r h} - 1} \epsilon_ {1} \\ \widetilde {\mathbf {x}} _ {t} = \frac {\sigma_ {t} ^ {2}}{\sigma_ {s} ^ {2}} \widetilde {\mathbf {x}} _ {s} - (e ^ {- 2 h} - 1) \left[ \left(1 - \frac {1}{2 r}\right) D _ {\theta} (\widetilde {\mathbf {x}} _ {s}; \sigma_ {s}) + \frac {1}{2 r} D _ {\theta} (\widetilde {\mathbf {x}} _ {1}; \sigma_ {s _ {1}}) \right] \\ + \sigma_ {t} \left[ \sqrt {e ^ {- 2 h} - e ^ {- 2 r h}} \epsilon_ {1} + \sqrt {e ^ {- 2 r h} - 1} \epsilon_ {2} \right]. \\ \end{array}
$$
B.2.7 The EDM case
In the EDM-preconditioned case [16, App. B.6], we set  $\sigma_t \coloneqq t$  and  $\alpha_t \coloneqq 1$ . We denote  $\sigma_d \coloneqq \sigma_{\mathrm{data}}$  the variance of the considered initial dataset and we set
$$
c _ {1} (t) = \frac {\sigma_ {d} ^ {2}}{t ^ {2} + \sigma_ {d} ^ {2}}, c _ {2} (t) = \frac {t \sigma_ {d}}{\sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}, c _ {3} (t) = \frac {1}{\sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}, c _ {4} (t) = \frac {1}{4} \log (t),
$$
so we obtain the following coefficients:
$$
A ^ {1} (t) = \frac {2}{t} \quad B ^ {1} (t) = - \frac {2}{t} \tag {DP NRSDE}
$$
$$
A ^ {2} (t) = \frac {2}{t} \left(1 - \frac {\sigma_ {d} ^ {2}}{t ^ {2} + \sigma_ {d} ^ {2}}\right) \quad B ^ {2} (t) = \frac {- 2 \sigma_ {d}}{\sqrt {t ^ {2} + \sigma_ {d} ^ {2}}} \quad (\mathrm {N P N R S D E})
$$
$$
A ^ {3} (t) = \frac {1}{t} \quad B ^ {3} (t) = - \frac {1}{t} \quad (\mathrm {D P N P F O})
$$
$$
A ^ {4} (t) = \frac {1}{t} \left(1 - \frac {\sigma_ {d} ^ {2}}{t ^ {2} + \sigma_ {d} ^ {2}}\right) \quad B ^ {4} (t) = \frac {- \sigma_ {d}}{\sqrt {t ^ {2} + \sigma_ {d} ^ {2}}} \quad (\mathrm {N P N P F O})
$$
In particular, the data prediction neural SDE/PFO are identical to those in the VE case with  $\sigma_t = t$ . So let us concentrate on the noise prediction regime, leading us to prove Proposition 3.2.
B.2.8 Proof of Proposition 3.2
In the Noise Prediction case, we have
$$
\Phi^ {2} (t, s) = \frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}}, \quad \Phi^ {4} (t, s) = \sqrt {\frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}}},
$$
so we readily compute:
$$
\int_ {s} ^ {t} \Phi^ {2} (t, \tau) B ^ {2} (\tau) \mathrm {d} \tau = (t ^ {2} + \sigma_ {d} ^ {2}) \int_ {s} ^ {t} \frac {1}{\tau^ {2} + \sigma_ {d} ^ {2}} \cdot \frac {- 2 \sigma_ {d}}{\sqrt {\tau^ {2} + \sigma_ {d} ^ {2}}} \mathrm {d} \tau ,
$$
$$
\int_ {s} ^ {t} \Phi^ {4} (t, \tau) B ^ {4} (\tau) \mathrm {d} \tau = \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \int_ {s} ^ {t} \frac {\sigma_ {d}}{\sqrt {\tau^ {2} + \sigma_ {d} ^ {2}}} \cdot \frac {- 1}{\sqrt {\tau^ {2} + \sigma_ {d} ^ {2}}} \mathrm {d} \tau .
$$
Let us consider two different changes of variables:
$$
\lambda_ {t} := - \log \left(\arctan \left(\frac {t}{\sigma_ {d}}\right)\right) \quad \text {a n d} \quad \lambda_ {t} := - \log \left(\frac {t}{\sigma_ {d} \sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}\right), \tag {37}
$$
that will be used for the (NP NPFO) and (NP NRSDE), respectively. For the former case, we have
$$
e ^ {- \lambda_ {t}} \mathrm {d} \lambda_ {t} = - \frac {\frac {1}{\sigma_ {d}} \mathrm {d} t}{1 + \frac {t ^ {2}}{\sigma_ {d} ^ {2}}} = - \frac {\sigma_ {d} \mathrm {d} t}{\sigma_ {d} ^ {2} + t ^ {2}}.
$$
Therefore, we can deduce that
$$
\begin{array}{l} \int_ {s} ^ {t} \Phi^ {4} (t; \tau) B ^ {4} (\tau) \mathrm {d} \tau = \int_ {s} ^ {t} \sqrt {\frac {t ^ {2} + \sigma_ {d} ^ {2}}{\tau^ {2} + \sigma_ {d} ^ {2}}} \cdot \frac {- \sigma_ {d}}{\sqrt {\tau^ {2} + \sigma_ {d} ^ {2}}} d \tau \\ = \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} \frac {- \sigma_ {d}}{\tau^ {2} + \sigma_ {d} ^ {2}} d \tau \\ = \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {- \lambda} d \lambda \\ = \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \arctan \left(\frac {t}{\sigma_ {d}}\right) (e ^ {h} - 1). \\ \end{array}
$$
For the latter case, we have
$$
\begin{array}{l} e ^ {- \lambda_ {t}} \mathrm {d} \lambda_ {t} = - \frac {\sigma_ {d} \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} - t \sigma_ {d} \frac {t}{\sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}}{\sigma_ {d} ^ {2} (t ^ {2} + \sigma_ {d} ^ {2})} \\ { = } { - \frac { \sigma _ { d } ^ { 2 } } { \sigma _ { d } ( t ^ { 2 } + \sigma _ { d } ^ { 2 } ) \sqrt { t ^ { 2 } + \sigma _ { d } ^ { 2 } } } = - \frac { \sigma _ { d } } { ( t ^ { 2 } + \sigma _ { d } ^ { 2 } ) \sqrt { t ^ { 2 } + \sigma _ { d } ^ { 2 } } } . } \\ \end{array}
$$
We then obtain
$$
\begin{array}{l} \int_ {s} ^ {t} \Phi^ {2} (t; \tau) B ^ {2} (\tau) \mathrm {d} \tau = \int_ {s} ^ {t} \frac {t ^ {2} + \sigma_ {d} ^ {2}}{\tau^ {2} + \sigma_ {d} ^ {2}} \cdot \frac {- 2 \sigma_ {d}}{\sqrt {\tau^ {2} + \sigma_ {d} ^ {2}}} \mathrm {d} \tau \\ = 2 \left(t ^ {2} + \sigma_ {d} ^ {2}\right) \int_ {s} ^ {t} e ^ {- \lambda} d \lambda \\ = \frac {2 t \sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}{\sigma_ {d}} (e ^ {h} - 1). \\ \end{array}
$$
The stochastic integral  $\int_s^t\Phi^2 (t,\tau)g(\tau)\mathrm{d}\bar{\omega}_\tau$  in noise prediction case is a Gaussian random variable with zero mean and whose variance can be computed by Ito isometry as
$$
\begin{array}{l} \int_ {t} ^ {s} [ \Phi^ {2} (t; \tau) g ^ {2} (\tau) ] ^ {2} \mathrm {d} \tau = (t ^ {2} + \sigma_ {d} ^ {2}) ^ {2} \int_ {t} ^ {s} \frac {1}{[ \tau^ {2} + \sigma_ {d} ^ {2} ] ^ {2}} 2 \tau \mathrm {d} \tau \\ = (t ^ {2} + \sigma_ {d} ^ {2}) ^ {2} \int_ {t} ^ {s} \frac {1}{[ \tau^ {2} + \sigma_ {d} ^ {2} ] ^ {2}} \mathrm {d} \tau^ {2} \\ = \left(t ^ {2} + \sigma_ {d} ^ {2}\right) ^ {2} \left(- \frac {1}{s ^ {2} + \sigma_ {d} ^ {2}} + \frac {1}{t ^ {2} + \sigma_ {d} ^ {2}}\right) \\ = \frac {(t ^ {2} + \sigma_ {d} ^ {2}) (s ^ {2} - t ^ {2})}{(s ^ {2} + \sigma_ {d} ^ {2})}. \\ \end{array}
$$
Putting everything together, we obtain the analytic solution at time  $t$  of (2) with coefficients (11) and initial value  $\mathbf{x}_s$  for the (NP NRSDE):
$$
\mathbf {x} _ {t} = \frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}} \mathbf {x} _ {s} + 2 \left(t ^ {2} + \sigma_ {d} ^ {2}\right) \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {- \lambda} \hat {F} _ {\theta} \left(\widehat {\mathbf {x}} _ {\lambda}, \lambda\right) d \lambda - \sqrt {2} \left(t ^ {2} + \sigma_ {d} ^ {2}\right) \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {- \lambda} d \bar {\omega} _ {\lambda}, \tag {38}
$$
where  $\lambda_{t} := -\log \left[\frac{t}{\sigma_{d}\sqrt{t^{2} + \sigma_{d}^{2}}}\right]$ . For the (NP NPFO), it is given by
$$
\mathbf {x} _ {t} = \sqrt {\frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}}} \mathbf {x} _ {s} + \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \int_ {\lambda_ {s}} ^ {\lambda_ {t}} e ^ {- \lambda} \hat {F} _ {\theta} (\hat {\mathbf {x}} _ {\lambda}, \lambda) d \lambda , \quad \lambda_ {t} := - \log \left[ \arctan \left[ \frac {t}{\sigma_ {d}} \right] \right]. \tag {39}
$$
Remark B.2. From the above proof, we immediately deduce the SEEDS-1 iterates in the EDM-preconditioned noise prediction case. These are given, for the (NP NPFO) and (NP NRSDE) respectively, by
$$
\widetilde {\mathbf {x}} _ {t} = \sqrt {\frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}}} \widetilde {\mathbf {x}} _ {s} + \sqrt {t ^ {2} + \sigma_ {d} ^ {2}} \arctan \left(\frac {t}{\sigma_ {d}}\right) (e ^ {h} - 1) \epsilon_ {\theta} (\widetilde {\mathbf {x}} _ {s}, s), \tag {40}
$$
$$
\widetilde {\mathbf {x}} _ {t} = \frac {t ^ {2} + \sigma_ {d} ^ {2}}{s ^ {2} + \sigma_ {d} ^ {2}} \widetilde {\mathbf {x}} _ {s} + \frac {2 t \sqrt {t ^ {2} + \sigma_ {d} ^ {2}}}{\sigma_ {d}} \left(e ^ {h} - 1\right) \epsilon_ {\theta} (\widetilde {\mathbf {x}} _ {s}, s) + \sqrt {\frac {\left(t ^ {2} + \sigma_ {d} ^ {2}\right) \left(s ^ {2} - t ^ {2}\right)}{\left(s ^ {2} + \sigma_ {d} ^ {2}\right)}} \epsilon , \tag {41}
$$
where  $\epsilon \sim \mathcal{N}(\mathbf{0},\mathbf{I}_d)$
B.2.9 The SEEDS algorithms in the EDM Noise Prediction case
In the same way we presented the SEEDS algorithms 3 and 4 for the VP case, the change-of-variables optimized for the EDM framework in Proposition 3.2 induces algorithms 5 for SEEDS-2 and 6 for SEEDS-3 respectively, under the Noise Prediction regime. This is the version of SEEDS that we use to achieve same performance as EDM solver but with twice less NFEs than the latter for ImageNet-64 (see Table 1 and Fig. 1 (c) in the main part of this article).
Algorithm 5 SEEDS-2 (EDM NP)
Input: initial value  $\mathbf{x}_T$ , steps  $\{t_i\}_{i=0}^M$ , data prediction model  $D_\theta$ ,  $r = 1/2$
Initialize  $\tilde{\mathbf{x}}_{t_0}\gets \mathbf{x}_T$
for  $i = 1$  to  $M - 1$  do
$$
(t, s) \leftarrow \left(t _ {i}, t _ {i - 1}\right), \quad \left(z ^ {1}, z ^ {2}\right) \leftarrow \mathcal {N} (0, \mathrm {I d}) ^ {\otimes 2}
$$