Global Well-posedness and Convergence Analysis of Score-based Generative Models via Sharp Lipschitz Estimates

## Abstract

Abstract We establish global well-posedness and convergence of the score-based generative models (SGM) under minimal general assumptions of initial data for score estimation. For the smooth case , we start from a Lipschitz bound of the score function with optimal time length. The optimality is validated by an example whose Lipschitz constant of scores is bounded at initial but blows up in finite time. This necessitates the separation of time scales in conventional bounds for non-log-concave distributions. In contrast, our follow up analysis only relies on a local Lipschitz condition and is valid globally in time. This leads to the convergence of numerical scheme without time separation. For the non-smooth case, we show that the optimal Lipschitz bound is O ⁢ ( 1 / t ) 𝑂 1 𝑡 O(1/t) italic_O ( 1 / italic_t ) in the point-wise sense for distributions supported on a compact, smooth and low-dimensional manifold with boundary.

## 1 Introduction

Diffusion models (DM) have become the state-of-the-art tools lately in generative AI
[, , ] such as image synthesis [, ].
DMs first evolve data samples with stochastic differential equation (SDE) to gradually inject Gaussian noise until a Gaussian distribution is reached. Then it approximates the drift in the associated backward (time-reversed) SDE and
generate a data sample from Gaussian noise.
The drift of the backward SDE contains
the gradient of the forward logarithmic density (score) that is estimated by solving a matching problem with deep neural network training. The reversibility concept of SDEs
dated back to Kolmogorov’s work [] in 1937, and the general score formula was derived by Anderson [] in 1982.

Theoretical study on the convergence of DM generated distribution to the target (data) distribution typically assumes that the data distribution admits a density with respect to Lebesgue measure [] among others. By also imposing that the score
of the data distribution is Lipschitz continuous, the score function of the forward process (the drift in the backward process) is well-behaved (not exploding) as the backward time tends to zero when the desired target sample is to be generated. However, this is not always observed in
practice and experimentally the score can blow up []. In particular, the explosion occurs at generation if the data distribution satisfies the manifold hypothesis (MH) [, ] which is verified for image data in []. Under MH, []
showed that the limit of the continuous backward process with approximate score is well-defined and that the sample distribution shares the same support as the target distribution under the integrability conditions on the error of score matching.
Also under MH, [] found quantitative bounds on the 1-Wasserstein distance between a compact target (data) distribution and the generative
distribution of DM by allowing the score function to explode as backward time approaches zero.

Both of the references ([]
and []), among others ([] for smooth case in score-based model, [] for Langevin MC, [] for ODE flows, [] for Föllmer flows), require a (locally) Lipschitz estimate for the score function to ensure the well-poseness of the backward SDEs and the approximation bound of the score matching and sampling process.

The goal of this paper is to provide sharp estimates that 1) confirm/improve the score assumptions of the existing convergence theory, 2) give insight for the duration of the forward process so that the backward process is well-defined, and 3) justify practical implementation of the backward process (e.g. early stopping strategies or truncation []).

#### Related work

We are aware of the convergence bound of discrete schemes for backward processes in []. Our convergence bound
takes the KL chain inequality (Proposition C.3.[]) as the building block. While equipped with sharp (local) Lipschitz bounds in the paper, we achieved polynomial complexity of sampling in the general smooth $p_{0}$ setting without separated regimes of schedule. We are also aware of [] which provides convergence bound in Wasserstein distance under a singular $p_{0}$ setting, supported on a compact manifold. Due to the potential singular behaviour of the score, early stopping schedules are employed []. Our paper provides sharp Lipschitz bounds of the singularities and therefore insights for the choices of schedules and loss normalization between discretization points.

The main contributions of this paper are:

- •
Realistic or sharp point-wise gradient and Hessian estimates of
the score potential function $\log p$ from commonly hypothesized data distributions.
- •
The first sharp example demonstrating the loss of Lipschtiz bound of the score function as time gets large even with nice initial data.
- •
Well-posedness and convergence of the backward diffusion process up to time zero (the generation time) in the smooth setting without separated regime of discretization.
- •
Characterization of the score (and its derivatives) in the setting of manifold hypothesis.

The rest of the paper is organized as follows. In Section 2, we first introduce settings of the diffusion model and discretization schemes of the backward process. Later, we present the transformation that relates the Fokker Planc equation with unbounded coefficients (density equation of forward process) to the non-linear Hamilton Jacobi equation and heat equation, which serves as the foundation of the analysis. The main theoretical results, Hessian estimate of score potential function $\log p$, are listed in Section 3. Based on these estimates, we establish well-posedness of the continous backward process and convergence bound of discretization in Section 4. The details of the proofs are in the Appendix.

## 2 Preliminaries

### 2.1 Background and Setting the Stage

A large class of generative diffusion models can be analyzed under the SDE framework []. It consists of two processes: forward and backward. *The forward process*, which relates to training, is an Ornstein-Uhlenbeck (OU) process in ${\mathbb{R}}^{n}$ as follows:

$$ $dX_{t}=-\frac{1}{2}X_{t}dt+dW_{t},\quad\text{for $t\in[0,T]$}$ (1) $$

where $W_{t}$ is a standard Brownian motion, $T$ is the final time such that the distribution of $X_{T}$ approximates a normal distribution in ${\mathbb{R}}^{n}$, namely $\mathcal{N}(0,I_{n})$. The initial distribution $X_{0}$ follows a target (data) distribution in ${\mathbb{R}}^{n}$ during the generative task, denoted as $p_{0}$. *The backward process*, which relates to generation of new data, is defined as an ’inversion’ of forward process (1). More precisely, with time reversal $t^{\prime}=T-t$,

$$ $d\tilde{X}_{t^{\prime}}=\left({\frac{1}{2}}\tilde{X}_{t^{\prime}}+\nabla\log p (T-t^{\prime},\tilde{X}_{t^{\prime}})\right)\,d{t^{\prime}}+d\tilde{W}_{t^{ \prime}}\quad\text{for $t^{\prime}\in[0,T]$},$ (2) $$

where $W_{t^{\prime}}$ is a standard Brownian motion (not necessarily being the same as $W_{t}$) and the initial distribution $\tilde{X}_{0}$ follows $\mathcal{N}(0,I_{n})$. The term $\nabla\log p$ is introduced in Eq. (2) such that the marginal distributions of the forward and backward processes are identical [].

To be specific, let $p:=p(x,t)$ denote the probability distribution function of the forward process (1), which solves the Fokker Planck equation with Cauchy data $p_{0}$, namely

$$ $\begin{cases}p_{t}=\frac{1}{2}(\nabla\cdot(xp)+\Delta\,p)\\ p(x,0)=p_{0}(x).\end{cases}.$ (3) $$

We also denote $P_{t}$ ($Q_{t}^{\prime}$ correspondingly) as the marginal distribution of $X_{t}$ in (1) ($\tilde{X}_{t^{\prime}}$ in (2)). Given initial distribution for (2) $Q_{0}\sim P_{T}$, then []: $\forall t$, $Q_{t}=P_{T-t}$. Especially, $Q_{T}=P_{0}$ so data $\sim p_{0}$ can be generated by solving (2).

In practice, since no closed form expression of $p_{0}$ is known, the $p$ in (3) is not analytically available.
Thus $\nabla\log p$ is approximated by a neural network $s:=s_{\theta}(t,x)$, where $\theta$ denotes latent variables of neural network and is omitted for simplicity of notation. The approximation is obtained by training the neural network with an $L_{2}$ score estimation loss, $\forall t\in[0,T]$,

$$ $\displaystyle\mathbb{E}_{x\sim P_{t}}||s_{\theta}(t,x)-\nabla\log p(t,x)||^{2}.$ $$

In the analysis, we assume an $\epsilon^{2}$ bounds for this estimation, see Assumption 2.1.

Given the approximation of score $s_{\theta}$, we employ the exponential scheme [] with initial distribution $\mathcal{N}(0,I_{n})$. More precisely, let $\delta=t_{0}\leq t_{1}\leq\cdots\leq t_{N}=T$ be the discretization points. $\delta=0$ for the normal setting and $\delta>0$ for the early-stopping setting. Then with $t_{k}^{\prime}=T-t_{N-k}$, the process in the discrete scheme is as follows:

$$ $\displaystyle d\hat{x}_{t^{\prime}}=(\frac{1}{2}\hat{x}_{t^{\prime}}+s_{\theta }(\hat{x}_{t_{k}^{\prime}},T-t_{k}^{\prime}))dt+d\hat{w}_{t^{\prime}}\quad t^{ \prime}\in[t_{k}^{\prime},t_{k+1}^{\prime}],\quad k=0,\cdots,N-1,$ (4) $$

which admits an explicit solution, with $\mu_{k}\sim\mathcal{N}(0,I_{n})$,

$$ $\displaystyle d\hat{x}_{t^{\prime}_{k+1}}=e^{\frac{1}{2}(t^{\prime}_{k+1}-t^{ \prime}_{k})}\hat{x}_{t^{\prime}_{k}}+2(e^{\frac{1}{2}(t^{\prime}_{k+1}-t^{ \prime}_{k})}-1)s_{\theta}(\hat{x}_{t_{k}^{\prime}},T-t_{k}^{\prime}))+\sqrt{e ^{(t^{\prime}_{k+1}-t^{\prime}_{k})}-1}\mu_{k}.$ (5) $$

Due to the limited knowledge of $p_{0}$ as well as the regularity of $\nabla\log p$, we restrict ourselves to uniform discretization points. Detailed selection is stated in the convergence theorems.

We assume the following bound of score approximation at the discretization points,

###### Assumption 2.1

Let $t_{k}$ be the discretization point of the scheme (4),

$$ $\displaystyle\frac{1}{T}\sum_{k=1}^{N}(t_{k}-t_{k-1})E_{x\sim P_{t_{k}}}\| \nabla\log p(t_{k},x)-s(t_{k},x)\|^{2}\leq\epsilon_{0}^{2}.$ (6) $$

###### Assumption 2.1

### 2.2 Foundational Ideas based on Non-linear Hamilton Jacobi Equation

The foundation of our analysis is investigating the behaviour of $\log p$ as the solution of a non-linear Hamilton Jacobi equation (HJE).
We consider the score potential function(^1^11Here we only consider the transform when the distribution of forward process $P_{t}$ is absolutely continuous with respect to Lebesgue measure. The transform and our analysis are valid for any $t>0$ in the general case and up to $t=0$ when $p_{0}$ is smooth.)

$$ $\displaystyle q(t,x)=-\log\,p(t,x)-\frac{|x|^{2}}{2}$ (7) $$

whose spatial gradient becomes the drift (score) in the backward (reverse time denoising and generation) process (2) of the diffusion model.
The $q$ function
satisfies the following PDE:

$$ $\begin{cases}q_{t}-\frac{1}{2}\Delta q+\frac{1}{2}(x\cdot\nabla q+|\nabla q|^{ 2})=0\\ q(0,x)=g(x),\end{cases}$ (8) $$

where $g(x)=-\log p_{0}(x)-|x|^{2}/2$, which is the non-Gaussian part of the likelihood function.

To simplify Eq.(8), we make a two step change of variables in time. First, let $\tilde{q}(t,x)=q(t,e^{t/2}x)$, then $\tilde{q}$ solves:

$$ $\displaystyle\tilde{q}_{t}=q_{t}+e^{t\over 2}x\cdot\nabla q(t,e^{t\over 2}x)= \frac{e^{-t}}{2}(\Delta\tilde{q}-|\nabla\tilde{q}|^{2}).$ (9) $$

Then we consider $\bar{q}(t,x)=\tilde{q}(-\log(1-t),x)$, then $\bar{q}$ solves:

$$ $\begin{cases}\bar{q}_{t}=\frac{1}{2}(\Delta\bar{q}-|\nabla\bar{q}|^{2})\quad t \in[0,1)\\ \bar{q}(0,x)=q_{0}\end{cases}.$ (10) $$

###### Remark 2.2

By a direct calculation

$$ $\displaystyle\bar{q}(t,x)=q\left(-\log(1-t),\ \frac{1}{\sqrt{1-t}}x\right) \text{ or equivalently, }q(t,x)=\bar{q}(1-e^{-t},e^{-t/2}x).$ (11) $$

Furthermore,

$$ $\displaystyle\nabla q(t,x)=e^{-t/2}\nabla\bar{q}(1-e^{-t},e^{-t/2}x)\text{ and , }\nabla^{2}q(t,x)=e^{-t}\nabla\bar{q}(1-e^{-t},e^{-t/2}x).$ (12) $$

Lastly, we also define $\bar{p}(t,x)=e^{-\bar{q}(t,x)}$, which satisfies

$$ $\begin{cases}\bar{p}_{t}={\frac{1}{2}}\Delta\bar{p}\quad\text{on $(0,1)\times{ \mathbb{R}}^{n}$}\\ \bar{p}(0,x)=h(x)=e^{-g(x)}.\end{cases}.$ (13) $$

The solution of (13) is given by

$$ $\bar{p}(x,t)={1\over(2\pi t)^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-|x-y|^{2} \over 2t}e^{-g(y)}\,dy.$ $$

To derive reasonable point-wise estimates of gradients and Hessian of the score function $q(t,x)$ that does not involve $1/t$, we will need the following assumption in relevant results. This assumption also ensures the above integration is well defined for $t\in[0,1]$.

###### Assumption 2.3

The tail distribution is bounded by some Gaussian distribution, i.e,

$$ $\displaystyle\log p_{0}(x)-\log p_{0}(0)\leq\alpha_{1}+\frac{1}{2}(\alpha_{2}- 1)|x|^{2}$ (14) $$

for constants
$\alpha_{2}<1$ and $\alpha_{1}\in R$. Without loss of generality we assume $\alpha_{2}>0$.

Recalling definition of $g$, it is equivalent to

$$ $\displaystyle g(x)-g(0)\geq-{\alpha_{2}\over 2}|x|^{2}-\alpha_{1},$ (15) $$

Technically speaking, the $g(0)$ could be absorbed into $\alpha_{1}$. We put it there just to track possible dependence on the dimension $n$.

Note that Assumption 2.3 implies that the second order moment of the process is bounded, i.e., $E_{p_{0}}||X||^{2}=M_{2}<\infty$.

###### Remark 2.2

###### Assumption 2.3

#### General notations

Throughout this paper, for an $n\times n$ matrix $A$, we use the spectral norm

$$ $||A||_{2}=\max_{\{v\in{\mathbb{R}}^{n}:\ |v|=1\}}|Av|=\text{the largest eigenvalue of $\sqrt{AA^{\top}}$}.$ (16) $$

In particular, for a map $F:{\mathbb{R}}^{n}\to{\mathbb{R}}^{n}$,

$$ $||\nabla F||_{2}\leq L\quad\Leftrightarrow\quad|F(x)-F(y)|\leq L|x-y|.$ $$

We also adopt the following notation when comparing two symmetric (Hessian) matrices,

$$ $\displaystyle A\preceq B\text{ if $B-A$ is semi-positive definite.}$ (17) $$

So for any symmetric matrix $A$, $\|A\|_{2}\leq\sigma\Leftrightarrow-\sigma I_{n}\preceq A\preceq\sigma I_{n}$.

## 3 Sharp Hessian Bound of Score Potential Function

The fundamental question, which is directly related to the well-posedness and convergence rate of the diffusion model [, ], is whether for any $T>0$, there exists a constant $C_{T}$ that depends only on $T$ and the initial data such that

$$ $\sup_{[0,T]\times{\mathbb{R}}^{n}}\|D_{x}^{2}q(t,x)\|_{2}\leq C_{T}\ ?$ (18) $$

In Section 3.1 we provide a uniform Hessian bound which only lasts for a finite time if the $p_{0}$ is not log-concave. Interestingly, the bound turns out to be optimal in sense of lasting time as in Example 3.4 we construct an initial distribution $p_{0}$ such that the Hessian of $\log p$ loses global bound right at the limiting time.
Inspired by the counter-example, alternatively in Section 3.2 we provide a locally Lipschitz estimate that lasts for $t\in[0,\infty)$. For the non-smooth case, in Section 3.3, we characterize the singular behaviour of $\log p$ and its derivatives.

### 3.1 Hessian Estimate of Score Potential Function in Finite Time

###### Theorem 3.1

Let $M_{0}$ be a nonnegative number. $g\in C^{2}({\mathbb{R}}^{d})$(^2^22The assumption is equivalent to $\log p_{0}\in C^{2}({\mathbb{R}}^{d})$.).

(1) If $D^{2}g(x)\preceq M_{1}I_{n}$, then

$$ $D_{x}^{2}q(t,x)\preceq e^{-{t}}M_{1}I_{n}\quad\text{for all $(t,x)\in[0,\infty )\times{\mathbb{R}}^{n}$}.$ $$

(2) If $D^{2}g(x)\succeq-M_{0}I_{n}$, then for any $T\in\left[0,-\log(1-\frac{1}{M_{0}})\right)$, we have

$$ $D_{x}^{2}q(t,x)\succeq-\frac{M_{0}}{e^{t}-M_{0}(e^{t}-1)}I_{n}\quad\text{for all $(t,x)\in(0,T]\times{\mathbb{R}}^{n}$}.$ $$

Note that if $M_{0}\leq 1$, then $T\in[0,\infty)$.

The proof is in Section C.1.As an immediate corollary, we have that

###### Corollary 3.2

Given data distribution $p_{0}\in C^{2}({\mathbb{R}}^{d})$ follows $-L_{1}I\preceq\sup_{x\in{\mathbb{R}}^{n}}D^{2}\log p_{0}(x)\preceq L_{0}I$. Then we have short time uniform bound of the Hessian: for any $t\in\left[0,-\log(1-\frac{1}{L_{0}+1})\right)$,

$$ $\sup_{{\mathbb{R}}^{n}}\|D_{x}^{2}\log p(t,x)\|_{2}\leq C_{t}.$ $$

where $C_{t}=\max\left(\frac{L_{0}+1}{1-(L_{0}+1)(e^{t}-1)}-1,\ e^{-t}(L_{1}-1)+1\right)$.

Furthermore, if $-\log p_{0}(x)$ is a convex function ($L_{0}\leq 0$), the estimate bound is global,

$$ $0\preceq-D_{x}^{2}\log p(t,x)\preceq(e^{-{t}}L_{1}+(1-e^{-t}))I_{n}\quad\text{ for all $(t,x)\in[0,\infty)\times{\mathbb{R}}^{n}$}.$ $$

###### Remark 3.3

The convex case has been also discussed in [] and it leads to single modal distribution. Similar finite bound was also derived in Lemma C.9 in [], which follows directly from the expression formula and the generalized Poincaré inequality for log-concave probability measures. Our Theorem 22 can be viewed as an improved version since the upper bound of $D^{2}\log p(t,x)$ only depends on the upper bound of $D^{2}\log p_{0}(x)$ instead of $||D^{2}\log p_{0}(x)||_{2}$ as in []. The PDE (partial differential equation) approach in our proof is more robust, which does not rely on the formula and could be easily adjusted to more general situations.

In the following, we show that the temporal bound $-\log\left(1-\frac{1}{M_{0}}\right)$ in the statements of Theorem 3.1 and Corollary 3.2 is sharp.

###### Example 3.4 (Loss of Uniform Hessian Bound)

There exists a smooth nonnegative $g$ satisfying assumptions in Theorem 3.1 and Corollary 3.2 ($M_{0}=M_{1}=2$) such that the corresponding $q(x,t)$ satisfies

$$ $\sup_{x\in{\mathbb{R}}^{n}}||D^{2}q\left(\log 2,x\right)||=\sup_{x\in{\mathbb{ R}}^{n}}||D^{2}\bar{q}\left({1/2},x\right)||=\infty.$ $$

Note that the number $1\over 2$ can be changed to any given time by re-scaling the function $\bar{q}(\lambda^{2}t,\lambda x).$ The detail of construction is in Section C.2

###### Remark 3.5

We are aware of some existing literature where similar finite time Hessian estimate is obtained given regularity of the initial data, for instance Lemma C.9. in [] and []. Our result, as stated in Theorem 3.1 is sharper in terms of longer valid time and consistency when $t\to 0$. More precisely, the key ingredient of the analysis in [] and in Theorem 3.1 is the ’semi-convexity’ of $-\nabla\log p$. For convex $p_{0}$, our analysis is valid for any compact interval of $[0,\infty)$ while [] lasts for $t$ such that, $e^{\frac{t}{2}}-e^{-\frac{t}{2}}\leq\frac{1}{2L},$ where $L$ is the Lipschitz constant of $\nabla\log p_{0}$. For non-convex $p_{0}$, taking example under construction, the Lipschitz constant $L=M_{0}+1$. Then the upper bound turns to $e^{\frac{t}{2}}-e^{-\frac{t}{2}}\leq\frac{1}{2M_{0}+2}$. In Theorem 3.1, it turns to

$$ $\displaystyle e^{-t}\geq 1-\frac{1}{M_{0}}$ $\displaystyle\Leftrightarrow(e^{\frac{t}{2}}-e^{-\frac{t}{2}})^{2}\leq\frac{1} {M_{0}(M_{0}-1)}\Leftarrow e^{\frac{t}{2}}-e^{-\frac{t}{2}}\leq\frac{1}{2M_{0} +2},$ (19) $$

which indicates our bound is better than [].
Furthermore, our estimate of upper bound of time does not require the upper bound of $D^{2}(-\log p)$.

###### Theorem 3.1

###### Corollary 3.2

###### Remark 3.3

###### Example 3.4 (Loss of Uniform Hessian Bound)

###### Remark 3.5

### 3.2 Local Estimate

The following theorem provides point-wise estimates of the score function, which can be quite useful in dealing with more general situations. Technically speaking, $g(x_{0})$ and $Dg(x_{0})$ can be absorbed into other parameters. Here we choose to display them to track the dependence of relevant parameters on the dimension $n$.

###### Theorem 3.6

Suppose that $\bar{p}=\bar{p}(t,x)$ is the solution to heat equation (13).
Let $|v|_{1}=\max\{|v|,1\}$ for $v\in{\mathbb{R}}^{n}$. Fix $x_{0}\in{\mathbb{R}}^{n}$.

(i) Given Assumption 2.3 and $|\nabla g(x)|\leq\beta_{1}|x-x_{0}|+\beta_{2}$ for $\beta_{1},\beta_{2}\geq 0$. Then for all $(t,x)\in[0,1]\times{\mathbb{R}}^{n}$,

$$ $|\nabla\bar{q}(t,x)|\leq{3\beta_{1}\over\sqrt{1-\alpha_{2}}}\max\{C_{n},\ C_{ \beta_{1},\alpha_{2}}|x-x_{0}|_{1}\}+\beta_{2}.$ (20) $$

Here the two constants $C_{n}=2\sqrt{(n+3)\log\left({2(1+4\beta_{1})\over\sqrt{1-\alpha_{2}}}\right)+4
n\log n+\alpha_{1}+1+{{{\beta_{2}}^{2}\over\beta_{1}}}}$ and $C_{\beta_{1},\alpha_{2}}=3\sqrt{\beta_{1}}+1+{6\alpha_{2}\over\sqrt{1-\alpha_{
2}}}$.

(ii) Assume $||D^{2}g(x)||_{2}\leq L$ and there exists $x_{0}$, $\alpha_{2}\in[0,1)$(^3^33In particular, if $g$ attains minimum at some point $x_{0}$, then we can take $\alpha_{2}=\alpha_{1}=0$.), such that

$$ $\displaystyle g(x)-g(x_{0})-\nabla g(x_{0})\cdot(x-x_{0})\geq-{\alpha_{2}\over 2 }|x-x_{0}|^{2}-\alpha_{1}\quad\forall x\in{\mathbb{R}}^{n}.$ (21) $$

Then for all $(t,x)\in[0,1]\times{\mathbb{R}}^{n}$,

$$ $||D^{2}\bar{q}(t,x)||\leq{10L^{2}+L\over 1-\alpha_{2}}\max\left\{\tilde{C}_{n} ^{2},\ (\tilde{C}_{L,\alpha_{2}})^{2}(|x-x_{0}-\nabla g(x_{0})|_{1})^{2}\right\},$ (22) $$

$$ $|\nabla\bar{q}_{t}(t,x)|\leq{48L^{2}+2L\over\sqrt{t}(1-\alpha_{2})^{3\over 2}} \max\left\{\tilde{C}_{n}^{3},\ (\tilde{C}_{L,\alpha_{2}})^{3}(|x-x_{0}-\nabla g (0)|_{1})^{3}\right\}.$ (23) $$

Here the two constants $\tilde{C}_{n}=2\sqrt{(n+3)\log\left({2(1+4L)\over\sqrt{1-\alpha_{2}}}\right)+4
n\log n+\alpha_{1}+1}$ and $\tilde{C}_{L,\alpha_{2}}=3\sqrt{L}+1+{6\alpha_{2}\over\sqrt{1-\alpha_{2}}}$.

The proof is in Section C.3. A simpler case with bounded $\nabla g$ is also discussed. See Remark C.2 for another way to bound $|\nabla\bar{q}_{t}(t,x)|$ by replacing $L^{2}\over\sqrt{t}$ in (23) by $O(nL^{3})$.

As an immediate corollary, we have

###### Corollary 3.7

Under the same assumption as (ii) in Theorem 3.6,
we further assume there exists $C_{0}>0$ such that, $\alpha_{1}\leq C_{0}n$ and $|\nabla g(x_{0})|\leq C\sqrt{n}$.
Then

$$ $\begin{array}[]{ll}&||D^{2}\bar{q}(t,x)||\leq CL^{2}\left(n\log n+L|x-x_{0}- \nabla g(x_{0})|^{2}\right)\\ &|\nabla\bar{q}_{t}(t,x)|\leq CL^{2}\left({(n\log n)}^{3\over 2}+L\sqrt{L}|x-x _{0}-\nabla g(x_{0})|^{3}\right).\end{array}$ $$

Here $C$ is a constant independent of $L$ and $n$.

Note that the further assumptions of scale relates to normalization in $n$ dimension.

###### Remark 3.8

Owing to 50 in the proof of Theorem 3.6), under the assumption of corollary 3.7, we have for all $m\in{\mathbb{N}}$

$$ $\mathbb{E}_{p(t,x)}(|x(t)|^{m})\leq O\left(L^{m\over 2}(n\log n)^{m\over 2}\right)$ $$

This demonstrates that, if we only care about expectations of powers of $D^{2}\bar{q}(t,x)$ or $\nabla\bar{q}_{t}(t,x)$, $\|D^{2}\bar{q}(t,x)\|_{2}$ behaves like $O(L^{3}n\log n)$ and $|\nabla\bar{q}_{t}(t,x)|$ behaves like $O\left(L^{3}\sqrt{L}(n\log n)^{3\over 2}\right)$. Note that the point $x_{0}$ itself plays no role in computing the expectation that is translation invariant in the $x$ variable.

###### Theorem 3.9

Let $g(x)\in C^{0,1}({\mathbb{R}}^{n})$ satisfy the Assumption 2.3 and $|\nabla g|\leq C(|x|+1)$ for a positive constant $C$. Then for any $T>0$, the above (2) is well-posed.

Proof: Note that $q(t,x)$ is a smooth function, hence locally Lipschitz continuous in $x$. Owing to Theorem 4.1, it suffices to show that for $q=-\log p(t,x)$,

$$ $|Dq(t,x)|\leq C_{T}(|x|+1)\quad\text{for all $(t,x)\in[0,T]\times{\mathbb{R}}^ {n}$}$ $$

for a constant $C_{T}$ depending on $C$ and $T$. By (11), it
is equivalent to showing that

$$ $|D\bar{q}(t,x)|\leq C_{T}(|x|+1)\quad\text{for all $(t,x)\in[0,1-e^{-T}]\times {\mathbb{R}}^{n}$}.$ $$

for a constant $C_{T}$ depending on $C$ and $T$, which follows from Theorem 3.6. $\square$

###### Theorem 3.6

###### Corollary 3.7

###### Remark 3.8

###### Theorem 3.9

### 3.3 Compactly Supported Data Distributions

In this section, we look at the situation where the data distribution $p_{0}$ is a positive measure with compact support which is a typical situation in image generation [].
Due to the manifold hypothesis, the support is typically a low dimension set. In this situation, what is important is the asymptotic estimate as $t\to 0$. The following are two estimates
related to [].

###### Theorem 3.10

We assume $\mathrm{supp}(p_{0})=D_{0}\subset\overline{B_{M}(0)}$ and the density is smoothly defined on $D_{0}$. We have,

$$ $\displaystyle\text{(1)}\quad|\nabla\bar{q}(t,x)|\leq{|x|+M\over t};$ (24) $\displaystyle\text{(2)}\quad||D^{2}\bar{q}(t,x)||_{2}\leq{1\over t}+{M^{2} \over t^{2}}.$ (25) $$

The proof is in Section C.4

Fixing $x$, denote by $\bar{y}_{t}$ the weighted center of mass:
$\bar{y}_{t}={\int_{D_{0}}ye^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}}$.
For a “regular" $\pi_{0}$, as $t\to 0$, we expect the measure $e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}$ will concentrate on $\{y\in D_{0}|\ |y-x|=d(x,D_{0})\}$. Thus,

$$ $\lim_{t\to 0}d(\bar{y}_{t},\Gamma_{x})=0,$ $$

where $\Gamma_{x}$ is the convex hull of $\{y\in D_{0}|\ |y-x|=d(x,D_{0})\}$. Then

$$ $|\nabla\bar{q}(t,x)|={|x-\bar{y}_{t}|\over t}\quad\mathrm{and}\quad\liminf_{t \to 0}t|\nabla\bar{q}(t,x)|\geq d(x,\Gamma_{x}).$ $$

So if $x\notin\Gamma_{x}$ (typical situation for low dimension set $D_{0}$), then $|D\bar{q}(t,x)|=O({1/t})$. Hence the $1/t$ blow up for the gradient bound is usually inevitable, which matches
experimental observations [].

Therefore, in practice, the initial distribution that we are really getting from the denoising process is $p(t_{0},x)$ for a small $t_{0}>0$. Per previous discussions, it is important to have a better global Lipschitz bound of $\log p(t_{0},x)$ than $O(1/t^{2})$. As a first step toward this direction, the following theorem says that the typical Hessian bound is actually $O\left(1/t\right)$ instead of $O\left(1/t^{2}\right)$ with the exception of a small set. Accordingly, in practice, it might be reasonable to assume that the Hessian is bounded by $O({1/t})$. For clarity of presentation, we will assume that $D_{0}$ is a low dimensional smooth manifold with boundary although our results can be properly extended to manifolds with less regularity. An example will be presented to show that our conclusion is optimal, see Example 3.12.

###### Theorem 3.11

For $1\leq d\leq n$, assume that $D_{0}\subset{\mathbb{R}}^{n}$ is a $d$-dimensional compact smooth manifold with boundary and $\pi_{0}$ is comparable to the uniform distribution on $D_{0}$. Then for almost everywhere $x\in{\mathbb{R}}^{n}$,

$$ $||D^{2}\bar{q}(t,x)||_{2}\leq{C_{x}\over t}\quad\text{for $t\in[0,1]$}.$ $$

Here $C_{x}$ is a constant depending only on $x$ and $D_{0}$. If $D_{0}$ is convex, then the above holds for all $x\in{\mathbb{R}}^{n}$.

Proof is in Section C.5

We will present a smooth non-convex $D_{0}$ that shows the result of Theorem 3.11 is optimal.

###### Example 3.12

Let $D_{0}\subset{\mathbb{R}}^{2}$ be the domain obtained by removing a small square $[0,2]\times[-1,1]$ from the big square $[-2,2]^{2}$ and then mollifying the corners to make it smooth. Here $O=(0,0)$. The $Y$-shaped region

$$ $L=\{x\in{\mathbb{R}}^{2}|\ \text{there are more than one $y$ such that $|x-y|= d(x,D_{0})$}\}.$ $$

We also choose $\pi_{0}={\chi_{D_{0}}\over|D_{0}|}dx$, i.e., the uniform distribution on $D_{0}$, where $|D_{0}|$ is the area of $D_{0}$.

Figure: Figure 1: In the above picture, $\Gamma_{x}=\{sy_{1}+(1-s)y_{2}:s\in[0,1]\}$

We have that,
$||D^{2}\bar{q}(x,t)||_{2}\geq{C_{x}/t^{2}}\quad\text{for $x\in L$ and $t\in(0,
1]$}$.

For reader’s convenience, we will verify the above when $x=(\theta,0)$ for $\theta>1$ in Section C.6. The other parts are left to interested readers as an exercise.

###### Theorem 3.10

###### Theorem 3.11

###### Example 3.12

## 4 Well-posedness and Convergence under Sharp Lipschitz Bound

As the starting point, we provide a well-posedness condition of a general SDE with additive noise.

###### Theorem 4.1

Given $T>0$, suppose that $F=F(t,x)\in C([0,T]\times{\mathbb{R}}^{n},{\mathbb{R}}^{n})$ satisfies that $F$ is locally Lipschitz continuous in $x$ variable, i.e., for any $M>0$, there exists a constant $L_{M}$ such that

$$ $|F(t,x)-F(t,y)|\leq L_{M}|x-y|\quad\text{for $x,y\in B_{M}(0)$ and $t\in[0,T]$}$ $$

and

$$ $|F(t,x)|\leq C(|x|+1).\quad\text{for $(t,x)\in[0,T]\times{\mathbb{R}}^{n}$}.$ (26) $$

for a positive constant C. For any $x_{0}\in{\mathbb{R}}^{n}$, the following SDE has a unique solution

$$ $dX_{t}=F(t,X_{t})dt+dW_{t},\quad\text{$t\in[0,T]$},\;\;X_{0}=x_{0}.$ $$

The proof is in Section C.7.

###### Remark 4.2

Note that the uniform Lipschitz continuity of $F$ is not needed given the linear growth condition (26), which is known to experts. See Theorem 2.4 and Theorem 3.1 in Chapter IV of [] for more general situations. For reader’s convenience, we presented the proof above for our special case.

Due to the limitation of global Hessian estimate, the convergence analysis is divided into the following two cases by $p_{0}$. The first case enjoys better complexity with respect to dimension ($N=\mathcal{O}(n)$) while has limitation in the final time $T$ in the forward process. The second case is valid globally in $T$ while achieving polynomial complexity ($N=\mathcal{O}(n^{3}\log^{2}n)$).

###### Theorem 4.1

###### Remark 4.2

#### Case I: p 0 subscript 𝑝 0 p_{0} italic_p start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT is (near) log-concave

###### Theorem 4.3

Assume the following global Hessian bound of $p_{0}$,

$$ $-L_{1}I_{n}\preceq D^{2}\log p_{0}(x)\preceq L_{0}I_{n}.$ $$

Let $\hat{q}_{T}$ be a distribution generated by the uniform discretization ($\delta=0$) of the exponential integrator scheme (4), with an approximated score satisfying the Assumption 2.1.
For $L_{1}>0$,

$$ $\displaystyle\mathrm{KL}(p_{0}\|\hat{q}_{T})\lesssim(M_{2}+d)e^{-T}+T\epsilon_ {0}^{2}+\frac{dT^{2}H_{T}^{2}}{N},$ $$

where $H_{T}$ depends on $L_{0}$, $L_{1}$, $T$ and $T<-\log(1-\frac{1}{L_{0}+1})$.

If $L_{1}\leq 0$, namely $p_{0}$ is log-concave,

$$ $\displaystyle\mathrm{KL}(p_{0}\|\hat{q}_{T})\lesssim(M_{2}+n)e^{-T}+T\epsilon_ {0}^{2}+\frac{nT^{2}H^{2}}{N},$ $$

where the constant $H$ depends on $L_{0}$ and $L_{1}$.

Proof: We first apply Corollary3.2 to attain global Hessian estimate in finite time. Then apply it to Theorem 4.1 for well-posedness and Proposition A.3 for convergence rate.

###### Theorem 4.3

#### Case II: General smooth p 0 subscript 𝑝 0 p_{0} italic_p start_POSTSUBSCRIPT 0 end_POSTSUBSCRIPT

###### Theorem 4.4

With the same assumption of Corollary 3.7, let $\hat{q}_{T}$ be a distribution generated by uniform discretization of the exponential integrator scheme (4), with an approximated score satisfying Assumption 2.1. Then we have,

$$ $\displaystyle\mathrm{KL}(p_{0}\|\hat{q}_{T})\lesssim(M_{2}+d)e^{-T}+T\epsilon_ {0}^{2}+{CL^{6}Tn(n\log n)^{2}\over N}.$ (27) $$

Proof: This is a direct consequence of Theorem C.5 to estimate truncation error in Proposition A.1.

In addition to the above cases, we also consider non-smooth $p_{0}$ supported on compact manifold. Restricted by the estimate in Theorem 3.10, we switch to the early stopping technique, namely $\delta>0$ in discretization. Due to the measure zero set (see in Section 3.3), the convergence bound is not yet optimal as shown in Section C.9.

###### Theorem 4.4

## 5 Conclusion

In this paper, we analyzed the Lipschitz bounds of the score in the SGM. Our bounds are sharp in light of the constructed counter-examples. Based on the result, we provide the guarantees for SGM in the framework where $L_{2}$ accurate score estimator is available and smoothness assumption holds on the data distribution. Our bounds for the non-smooth case characterize singular behaviours of the score near the generation time, offering insights for model parameterization in practice.

#### Limitation

Due to the limited knowledge of regularity factors of data distribution (e.g. optimal Lipschitz constants), our bound cannot provide implementable guidance on seeking the optimal schedule (which may require a separation of temporal regimes). Also in the manifold case, due to the complex geometries, as shown in the non-smooth section, our theories cannot provide a justifiable guidance of early stopping time. We will investigate these issues in a future study.

## Appendix A Convergence theories

In this section, we list some numerical algorithms and convergence theories related to the numerical discretization of (2). They are due to [].

#### Convergence in distribution

The key ingredient of the convergence theory is the following result from the chain rule of KL divergence.

###### Proposition A.1 (Prop C.3. of [ 7 ] )

Given the score error estimation Assumption 2.1, the exponential integrator scheme (4) satisfies,

$$ $\displaystyle\mathrm{KL}(p_{\delta}\|\hat{q}_{T-\delta})\lesssim\mathrm{KL}(p_ {T}\|\gamma_{n})+T\epsilon_{0}^{2}+\sum_{k=1}^{N}\int_{t_{k-1}}^{t_{k}}E\| \nabla\log p_{t}(x_{t})-\nabla\log p_{t_{k}}(x_{t_{k}})\|^{2}{\mathrm{d}}t,$ $$

where $\gamma_{n}$ is the Uniform Gaussian distribution

The first term $\mathrm{KL}(p_{T}\|\gamma_{n})$ in the estimate measures the distance between the measure of OU process to its invariant measure. When the data has finite second order moment, it tends to 0 0 as $T\to\infty$.

###### Proposition A.2 (Lem C.4 of [ 7 ] )

With finite second order moment $E_{p_{0}}|X|^{2}<\infty$, for $T>1$,

$$ $\displaystyle KL(p_{T},\gamma_{n})\leq(n+M_{2})e^{-T}.$ $$

The third term relates to local truncation error that depends on the regularity of the forward process. Then Proposition A.1 can be further extended if the global Hessian estimate is available.

###### Proposition A.3 (Theorem 2.1 of [ 7 ] )

Given assumption of Proposition A.1 , $\nabla\log p_{t}$ is $L$-Lipschitz. For uniform discretization, the exponential integrator scheme (4) satisfies,

$$ $\displaystyle\mathrm{KL}(p_{0}\|\hat{q}_{T})\lesssim(M_{2}+n)e^{-T}+T\epsilon_ {0}^{2}+\frac{nT^{2}L^{2}}{N}.$ $$

###### Proposition A.1 (Prop C.3. of [ 7 ] )

###### Proposition A.2 (Lem C.4 of [ 7 ] )

###### Proposition A.3 (Theorem 2.1 of [ 7 ] )

## Appendix B Semi-convexity of second order differential equation

Here we list some important theories for constructing the finite time log-convexity of the density of forward process.

Given a function $w(t,x)$, its convex envelop $w^{**}(t,x)$ in $x$ is defined as

$$ $\displaystyle w^{**}(t,x)=\inf\left\{\sum_{i=1}^{n+1}\lambda_{i}w(t,z_{i})|\ x =\sum_{i=1}^{n+1}\lambda_{i}z_{i},\ \sum_{i=1}^{n+1}\lambda_{i}=1,\ \lambda_{i }\geq 0,z_{i}\in{\mathbb{R}}^{n}\right\}.$ (28) $$

###### Lemma B.1 (Prop 7 of [ 1 ] , Lemma 2 of [ 22 ] )

Let $w$ be a solution of

$$ $\displaystyle w_{t}+F(t,x,\nabla w,D^{2}w)=0.$ (29) $$

The convex envelope $w^{**}$ of $w$ is a supersolution of (29), under the following assumptions,

- 1.
$F$ is elliptic in the sense $F(t,x,p,A)\geq F(t,x,p,\tilde{A})$ if $A\leq\tilde{A}$.
- 2.
$(x,A)\in{\mathbb{R}}^{n}\times S_{++}^{n}\mapsto F(t,x,p,A^{-1})$ concave for all $t$ and $p$. Here $S_{++}^{n}$ is the set of all $n\times n$ positive definite matrices.
- 3.
$w$ is coercive in the sense that
$\displaystyle\lim_{x\to\infty}\frac{w(t,x)}{|x|}=\infty,$
(30)
uniformly in $t$.

###### Lemma B.1 (Prop 7 of [ 1 ] , Lemma 2 of [ 22 ] )

## Appendix C Proofs

In this section, we collects the proofs to the results.

### C.1 Proof of Theorem 3.1

For (1), it is equivalent to showing that

$$ $D_{x}^{2}\bar{q}(t,x)\leq M_{1}I_{n}\quad\text{for all $(t,x)\in[0,1]\times{ \mathbb{R}}^{n}$}.$ $$

This actually follows immediate from (51). Here we will present a standard PDE approach that does not rely on the formula.

Let $\xi$ be a given unit vector. By taking derivatives of (10), we deduce that $v=q_{\xi\xi}$ satisfies

$$ $v_{t}-{\frac{1}{2}}\Delta v+\nabla\bar{q}\cdot\nabla v=-|\nabla\bar{q}_{\xi}|^ {2}\leq 0\quad\text{on $(0,1)\times{\mathbb{R}}^{n}$}.$ $$

Thanks to the standard maximum principle of parabolic equation, we have that

$$ $v(t,x)\leq\sup_{x\in{\mathbb{R}}^{n}}v(x,0)=\sup_{x\in{\mathbb{R}}^{2}}q_{\xi \xi}\leq M_{1}.$ $$

The proof of (2) is more interesting. It is equivalent to showing that

$$ $D^{2}\bar{q}(t,x)\geq-\frac{M_{0}}{1-M_{0}t}I_{n}.$ (31) $$

Below we show a PDE approach based on a modification of the arguments in [] for obtaining semiconcavity of solutions to the general viscous Hamilton-Jacobi equations.

Fix $\delta_{1}>0$. Note that by $D^{2}g\succeq-M_{0}I_{n}$,

$$ $g(x)-g(0)-\nabla g(0)\cdot x\geq-{M_{0}\over 2}|x|^{2}.$ $$

Hence there exists a constant $C_{\delta_{1}}$ such that

$$ $g(x)\geq-\left({M_{0}\over 2}+{\delta_{1}}\right)|x|^{2}-C_{\delta_{1}}.$ $$

Let $\alpha$ and $c$ be positive numbers satisfying

$$ $\theta(0)=\alpha\tan(\alpha c)\geq M_{\delta_{1}}={M_{0}}+4{\delta_{1}}$ (32) $$

Consider the following construction,

$$ $\displaystyle w=\bar{q}+\theta(t)|x|^{2}/2+n\Theta(t)/2,$ (33) $$

where,

$$ $\displaystyle\theta(t)=\alpha\tan(\alpha c+\alpha t),\quad t<T^{*}=\frac{\pi}{ 2\alpha}-c$ (35) $\displaystyle\Theta(t)=\int_{0}^{t}\theta(s)ds.$ (36) $$

We notice that Eq.(35) implies, $\theta(0)=\alpha\tan(\alpha c)$ and $\theta^{\prime}-\theta^{2}=\alpha^{2}$.
Then $w$ satisfies the following equation:

$$ $\displaystyle 0=w_{t}+F(t,x,\nabla w,\nabla^{2}w):=w_{t}-\frac{1}{2}\Delta w+ \frac{1}{2}|\nabla w|^{2}-\theta(t)\nabla w\cdot x-\frac{\alpha^{2}}{2}|x|^{2}.$ (37) $$

Now we consider the convex envelope (definition see (28)) of $w$, $w^{**}$ and aim to apply Lemma B.1 to show that $w^{**}$ is a supersolution of Eq.(37). After direct validation of the first two conditions of Lemma B.1, it resorts to coercivity assumption (30). To this end, we construct a solution $\underline{q}$ of the equation ((10)) subjecting to $\underline{q}(0,x)\leq\bar{g}(x)$:

$$ $\underline{q}=\theta_{1}(t)\frac{|x|^{2}}{2}+\Theta_{1}(t)\frac{n}{2}-C_{ \delta_{1}}$ (38) $$

where

$$ $\theta_{1}(t)=\frac{1}{t-\frac{1}{M_{0}+2{\delta_{1}}}},\quad\Theta_{1}(t)= \int_{0}^{t}\theta_{1}(t)dt.$ (39) $$

From Eq.(39), we know the construction holds for $t\in[0,{1\over M_{0}+2{\delta_{1}}})$.
By revisiting (35),

$$ $\sup_{\{\alpha\tan(\alpha c)\geq M_{\delta_{1}}\}}\left(\frac{\pi}{2\alpha}-c \right)=\lim_{\{\alpha\to 0^{+},\ \alpha\tan(\alpha c)=M_{\delta_{1}}\}}\left( \frac{\pi}{2\alpha}-c\right)={1\over M_{\delta_{1}}}={1\over M+4{\delta_{1}}}$ (40) $$

and

$$ $\displaystyle\lim_{\{\alpha\to 0^{+},\ \alpha\tan(\alpha c)=M_{\delta_{1}}\}} \theta(t)=\frac{M_{\delta_{1}}}{1-M_{\delta_{1}}t}.$ (41) $$

Then by choosing suitable $\alpha$ and $c$, the comparison principle of (10) which is equivalent to the one for (13) says that

$$ $\bar{q}(x,t)\geq\underline{q}(x,t)\quad\text{for $t\in[0,{1\over M_{\delta_{1} }})$}$ $$

and hence,

$$ $\displaystyle w\geq(\theta(t)+\theta_{1}(t))\frac{|x|^{2}}{2}+(\Theta(t)+ \Theta_{1}(t))\frac{n}{2}\quad\text{for $t\in[0,{1\over M_{\delta_{1}}})$}.$ (42) $$

Now turning back to Eq.(42), we know $\theta(t)+\theta_{1}(t)\geq 2{\delta_{1}}>0$ uniformly in any closed subinterval of $t\in[0,{1\over M_{\delta_{1}}})$. This ensures the uniform coercivity requirement in Lemma B.1.

Summing up, by Lemma B.1, $w^{**}$ is a supersolution. On the other side, as convex envelope, $w^{**}\leq w$. Next, we want to utilize the comparison principle of (37) to show $w^{**}\geq w$ for all $t$, which is equivalent to Eq.(10) due to the construction (33). To do this, we only need $w^{**}(0,X)\geq w(0,X)$, equivalently $w(0,X)$ is convex and is assured by $\theta(0)\geq M_{\delta_{1}}>M_{0}$.

Now we have $w=w^{**}$ for $x\in R^{d}$, $t\in[0,T^{*}]$, is convex. In particular, this implies that

$$ $D^{2}\bar{q}(t,x)\geq-\theta(t)I_{n}\quad\text{for $(t,x)\in[0,T^{*}]\times{ \mathbb{R}}^{n}$}.$ $$

Hence we derive that for any $T<{1\over M_{\delta_{1}}}$,

$$ $\displaystyle\inf_{[0,T]\times{\mathbb{R}}^{n}}D^{2}\bar{q}(t,x)\geq-\frac{M_{ \delta_{1}}}{1-M_{\delta_{1}}t}I_{n}.$ (43) $$

Then (31) follows by sending ${\delta_{1}}\to 0$.

Since the transform (11) only requires estimate of $\bar{q}$ for $t\in[0,1]$, when $M_{0}<1$, (43) holds for any $T<1$. Recalling transformation of $q$, the condition $M_{0}<1$ is equivalent to $-\log p$ is convex.

$\square$

### C.2 Construction of Example of the Loss of Uniform Hessian Bound in Section 3.4

Precisely speaking, we will construct a one dimensional ($n=1$) example of $g(x)=G^{2}(x)$ for a smooth Lipschitz continuous function $G$ satisfying that $|G^{\prime}(x)|\leq 1$, $|g^{\prime\prime}(x)|\leq 2=M_{0}=M_{1}$ and

$$ $\limsup_{|x|\to+\infty}\left|\bar{q}^{\prime\prime}\left(1/2,\ x\right)\right| =\infty.$ $$

For $M>0$, let $g_{M}$ be the even function such that

$$ $g_{M}(x)=\begin{cases}2M^{2}-x^{2},\quad 0\leq x\leq M\\ (x-2M)^{2},\quad M\leq x\leq 2M\\ 0,\quad x\geq 2M.\end{cases}$ $$

Note that $|g_{M}^{\prime\prime}|\leq 2$ independent of $M$. Let $h_{M}(t,\ x)$ be the solution to the following heat equation

$$ $\displaystyle u_{t}-{\frac{1}{2}}\Delta u=0\quad\text{for $(t,x)\in(0,\infty) \times{\mathbb{R}}$}$ (44) $$

subject to $h_{M}(0,x)=e^{-g_{M}(x)}$.

Since $h_{M}$ is even in $x$, $(h_{M})_{x}(1/2,\ 0)=0$. Hence, we have

$$ $\displaystyle(\log h_{M})_{xx}(1/2,\ 0)=\frac{(h_{M})_{xx}(1/2,\ 0)}{h_{M}(1/2 ,\ 0)}$ $\displaystyle=\frac{e^{-2M^{2}}\int_{0}^{M}(4y^{2}+2)\,dy+e^{-2M^{2}}\int_{M}^ {2M}e^{-2(y-M)^{2}}(4(y-2M)^{2}-2)\,dy}{e^{-2M^{2}}M+e^{-2M^{2}}\int_{M}^{2M}e ^{-2(y-M)^{2}}\,dy+\int_{2M}^{\infty}e^{-y^{2}}\,dy}$ $\displaystyle:=\frac{A+B}{C+D+E}.$ $$

Clearly, for $M\geq 1$,

$$ $A+B=4e^{-2M^{2}}\int_{0}^{M}y^{2}\,dy+4e^{-2M^{2}}\int_{M}^{2M}e^{-2(y-M)^{2}} (y-2M)^{2}\,dy>M^{3}e^{-2M^{2}}.$ $$

and

$$ $C+D+E\leq 3Me^{-2M^{2}}.$ $$

Thus

$$ $(\log h_{M})_{xx}(1/2,0)>{M^{2}\over 3}$ (45) $$

for all $M\geq 1$.

###### Remark C.1

Note that $g_{M}$ is $C^{1,1},$ not smooth. However, the estimates above still hold for sufficiently fine mollifications of $g_{M}$, so we may assume without loss of generality that $g_{M}$ is smooth.

Below we will choose a sequence $0\leq x_{1}\leq x_{2}\leq...$ such that the terms in $g:=\sum_{k=1}^{\infty}g_{k}(x-x_{k})$ have disjoint support, and such that

$$ $(\log\bar{p}_{N})_{xx}(1/2,\ x_{k})>{k^{2}\over 3},\quad\text{for $k=1,\,..., \,N$},$ (46) $$

where $\bar{p}_{N}$ is the solution to (44) subject to $\bar{p}_{N}(0,x)=e^{-\sum_{k=1}^{N}g_{k}(x-x_{k})}$. Note that $\bar{p}_{N}(0,x)=1$ for $x\leq-2$.

Suppose we have managed to do this. Note that

$$ $\hat{p}\leq\bar{p}_{N}\leq 1$ $$

where $\hat{p}$ is the solution to (44) subject to $\hat{p}(0,x)=\chi_{[-3,-2]}$. Then interior derivative estimates for solutions of (44) imply that $(\log\bar{p}_{N})_{xx}(1/2,\ \cdot)$ converges locally uniformly on $\mathbb{R}$ as $N\rightarrow\infty$ to $(\log\bar{p})_{xx}(1/2,\ \cdot,)$, where $q$ is the caloric function with initial data $e^{-g}$. Therefore, $(\log\bar{p})_{xx}(\cdot,\,0)$ is bounded, but $(\log\bar{p})_{xx}(1/2,\ \cdot)$ is unbounded (its values at $x_{k}$ are at least $k^{2}\over 3$), as desired. In addition, it is easy to check $\int_{\mathbb{R}}p(0,x)dx=\int_{\mathbb{R}}\exp(-g(x)-\frac{x^{2}}{2})dx<\int_
{\mathbb{R}}\exp(-\frac{x^{2}}{2})dx<\infty$. This implies the constructed $g$ corresponds to a density function after the transform (11) and appropriate normalization.

We now explain how to choose $x_{k}$. We will repeatedly use the fact that if $h$ is a bounded smooth function on $\mathbb{R}$ and $\tilde{h}$ is a compactly supported smooth function, then the caloric function with initial data $h+\tilde{h}(\cdot+S)$ converges in $C^{2}$ as $|S|\rightarrow\infty$ on compact subsets of $\{t>0\}$ to the caloric function with initial data $h$.

First, we let $x_{1}=0$. Then (46) with $N=1$ follows immediately from (45). Now suppose we have chosen $x_{1}<...<x_{M-1}$ such that the supports of $g_{k}(x-x_{k}),\,1\leq k\leq M-1$ are disjoint and (46) holds for $N=M-1$. Using the above-mentioned fact and (45), if we take $x_{M}$ sufficiently large, then (46) holds for $N=M$. Indeed, the inequality for $k<M$ follows immediately from the fact above, and the inequality for $k=M$ follows from the fact above and the inequality (45), after translating so that $x_{M}$ becomes 0 0. This completes the construction.

###### Remark C.1

### C.3 Proof of Theorem 3.6

Without loss of generality, we may assume $x_{0}=0$. It suffices to show the above for $|x|\geq 1$. For $|x|\leq 1$, we can just replace $|x|$ in all the final bounds with 1.

First, we prove (20). Without loss of generality, let $g(0)=0$. Then

$$ $g(z)\leq{\beta_{1}\over 2}|z|^{2}+\beta_{2}|z|\quad\text{for $z\in{\mathbb{R}} ^{n}$}.$ $$

Recall that

$$ $\begin{array}[]{ll}\bar{p}(t,x)&={1\over{(2\pi t)}^{n\over 2}}\int_{{\mathbb{R }}^{n}}e^{-{|x-y|^{2}\over 2t}}h(y)\,dy\[14.22636pt] &={1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}h(x-\sqrt{2t} y)\,dy.\end{array}$ (47) $$

Then

$$ $\begin{array}[]{ll}-\nabla\bar{q}={\nabla\bar{p}\over\bar{p}}&={1\over\bar{p}} {1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}h(x-\sqrt{2t}y) Dg\,dy\[14.22636pt] &={1\over\bar{p}}{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2} }}h(x-\sqrt{2t}y)Dg\,dy\end{array}$ (48) $$

Since $ab\leq a^{2}+{b^{2}\over 2}$,

$$ $g(x-\sqrt{2t}y)\leq\beta_{1}(|x|^{2}+2|y|^{2})+\beta_{2}(|x|+\sqrt{2}|y|)\leq 2 \beta_{1}(|x|^{2}+2|y|^{2})+{{\beta_{2}}^{2}\over\beta_{1}},$ $$

we deduce that

$$ $\begin{array}[]{ll}\bar{p}(x,t)&\geq e^{-{{\beta_{2}}^{2}\over\beta_{1}}}{e^{- 2\beta_{1}|x|^{2}}\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-(1+4\beta _{1}){|y|^{2}}}\,dy\[14.22636pt] &=e^{-{{\beta_{2}}^{2}\over\beta_{1}}}{e^{-2\beta_{1}|x|^{2}}}({1\over 1+4 \beta_{1}})^{n\over 2}.\end{array}$ (49) $$

Then

$$ $\begin{array}[]{ll}|\nabla\bar{p}|&={1\over{(\pi)}^{n\over 2}}\left|\int_{{ \mathbb{R}}^{n}}e^{-{|y|^{2}}}Dh(x-\sqrt{2t}y)\,dy\right|={1\over{(\pi)}^{n \over 2}}\left|\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}hDg(x-\sqrt{2t}y)\,dy \right|\[14.22636pt] &\leq{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}(\beta_{1}|x|+\sqrt{2} \beta_{1}|y|+\beta_{2})e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy\[14.22636pt] &=(\beta_{1}|x|+\beta_{2})\bar{p}+{\sqrt{2}\beta_{1}\over{(\pi)}^{n\over 2}} \int_{{\mathbb{R}}^{n}}|y|e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy.\end{array}$ $$

Let

$$ $\tilde{K}={2\over\sqrt{1-\alpha_{2}}}\max\left\{{C_{n,m}\over|x|},\ 4\sqrt{ \beta_{1}}+1\right\},$ $$

where

$$ $C_{n,m}={2}\sqrt{(n+3)\log\left({2(1+4\beta_{1})\over\sqrt{1-\alpha_{2}}} \right)+4n\log n+\alpha_{1}+J_{m}+{{{\beta_{2}}^{2}\over\beta_{1}}}}.$ $$

Here for $m\in{\mathbb{N}}$, $J_{m}$ is the last positive integer such that $e^{r^{2}}\geq r^{m}$ for when $r\geq J_{m}$. In particular, $J_{1}=J_{2}=J_{3}=1$ and $C_{n}=C_{n,m}$ for $m=1,2,3$.

Claim: If

$$ $K\geq K_{0}=\max\left\{\tilde{K},\ {6\alpha_{2}\over 1-\alpha_{2}}\right\},$ $$

then for $i=1,2,3,...,m$,

$$ $T_{i}(x)={1\over\bar{p}(t,x)}{1\over{(\pi)}^{n\over 2}}\int_{y\in{\mathbb{R}}^ {n}}|y|^{i}e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy\leq K^{i}|x|^{i}+1\leq 2K^{i}|x|^ {i}.$ (50) $$

Clearly, $T_{i}(0)=\mathbb{E}_{p(t,z)}(|z|^{i})$.

Let us prove the claim. Note that

$$ $\begin{array}[]{ll}\bar{p}(t,x)T_{i}&={1\over{(\pi)}^{n\over 2}}\int_{|y|\leq K |x|}|y|^{i}e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy+{1\over{(\pi)}^{n\over 2}}\int_{| y|\geq K|x|}|y|^{i}e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy\[14.22636pt] &\leq K^{i}|x|^{i}\bar{p}(x,t)+\underbrace{{1\over{(\pi)}^{n\over 2}}\int_{|y| ^{i}\geq K|x|}|y|^{i}e^{-{|y|^{2}}}h(x-\sqrt{2t}y)\,dy}_{\mathrm{I_{i}}}.\end{array}$ $$

Our goal is to show that $I_{i}\leq\bar{p}(t,x)$ when $K\geq K_{0}$. Since $g(z)\geq-\alpha_{2}|z|^{2}-\alpha_{1}$, $|y|\geq K|x|$ and $K>{6\alpha_{2}\over 1-\alpha_{2}}$,

$$ $\begin{array}[]{ll}g(x-\sqrt{2t}y)&\geq-{\alpha_{2}\over 2}\left(|x|+\sqrt{2}| y|\right)^{2}-\alpha_{1}\[14.22636pt] &>-\alpha_{2}|y|^{2}\left({1\over K}+1\right)^{2}-\alpha_{1}\[14.22636pt] &>\alpha_{2}|y|^{2}\left({3\over K}+1\right)-\alpha_{1}\[14.22636pt] &\geq-{(1+\alpha_{2})\over 2}|y|^{2}-\alpha_{1}\end{array}$ $$

Then

$$ $\mathrm{I_{i}}\leq{e^{\alpha_{1}}\over(\sqrt{\pi})^{n}}\int_{|y|\geq K|x|}|y|^ {i}e^{-{(1-\alpha_{2})|y|^{2}\over 2}}\,dy.$ $$

For convenience, denote $K_{1}={K\sqrt{1-\alpha_{2}}\over 2}$. Then

$$ $\begin{array}[]{ll}{e^{\alpha_{1}}\over(\sqrt{\pi})^{n}}\int_{|y|\geq K|x|}|y| ^{i}e^{-{(1-\alpha_{2})|y|^{2}\over 2}}\,dy&={e^{\alpha_{1}}\over(\sqrt{\pi})^ {n}}\left({2\over\sqrt{1-\alpha_{2}}}\right)^{n+i}\int_{|z|\geq{K_{1}|x|}}|z|^ {i}e^{-2|z|^{2}}\,dz\[14.22636pt] &\leq{e^{\alpha_{1}}\over(\sqrt{\pi})^{n}}\left({2\over\sqrt{1-\alpha_{2}}} \right)^{n+3}\int_{|z|\geq{K_{1}|x|}}e^{-|z|^{2}}\,dz.\end{array}$ $$

The last inequality is due to $e^{{r^{2}}}\geq r^{m}$ if $r\geq J_{m}$ and $K_{1}|x|\geq J_{m}$.

Note

$$ $\left(\int_{-r}^{r}e^{-t^{2}}\,dt\right)^{2}\geq\int_{\{{w\in{\mathbb{R}}^{2}| \ |y|\leq r}\}}e^{-|w|^{2}}\,dw=\pi\left(1-{e^{-r^{2}}}\right).$ $$

Combining with $(1-t)^{n}\geq 1-nt$ for $t\in[0,1]$, we deduce

$$ $\int_{Q_{r}}e^{-{|y|^{2}}}\,dy=\left(\int_{-r}^{r}e^{-t^{2}}\,dt\right)^{n} \geq(\sqrt{\pi})^{n}\left(1-{e^{-r^{2}}}\right)^{n\over 2}\geq(\sqrt{\pi})^{n} \left(1-{ne^{-r^{2}}\over 2}\right)$ $$

Here $Q_{r}=\{y=(y_{1},y_{2},...,y_{n})|\ |y_{i}|\leq r\ \text{for $i=1,2,...,n$}\}$. Accordingly, for $r\geq 1$,

$$ $\begin{array}[]{ll}{1\over(\sqrt{\pi})^{n}}\int_{|y|\geq r}e^{-{|y|^{2}}}\,dy& \leq{1\over(\sqrt{\pi})^{n}}\int_{{\mathbb{R}}^{n}\backslash Q_{r}}e^{-{|y|^{2 }}}\,dy+{1\over(\sqrt{\pi})^{n}}\int_{Q_{r}\backslash\{|y|\leq r\}}e^{-{|y|^{2 }}}\,dy\[14.22636pt] &\leq{ne^{-r^{2}}\over 2}+{1\over(\sqrt{\pi})^{n}}e^{-r^{2}}(2r)^{n}<2^{n}e^{- r^{2}}r^{n}\end{array}$ $$

This implies that

$$ ${1\over(\sqrt{\pi})^{n}}\int_{|z|<{K_{1}|x|}}e^{-|z|^{2}}\leq 2^{n}e^{-K_{1}^{ 2}|x|^{2}}\left(K_{1}|x|\right)^{n}.$ $$

Since $K_{1}\geq\max\{3\sqrt{\beta_{1}},{2\sqrt{n\log n}\over|x|}\}$,

$$ ${e^{-2\beta_{1}|x|^{2}}}K_{1}^{n}|x|^{n}e^{-{K_{1}^{2}|x|^{2}}}\leq{e^{-2\beta _{1}|x|^{2}}}e^{-{K_{1}^{2}|x|^{2}\over 2}}\leq e^{-{K_{1}^{2}|x|^{2}\over 4}}$ $$

Combining with

$$ ${K_{1}^{2}|x|^{2}\over 4}\geq{\tilde{K}^{2}|x|^{2}\over 4}\geq(n+3)\log\left({ 1+4\beta_{1}\over\sqrt{1-\alpha_{2}}}\right)+16n\log n+\alpha_{1}+1+{{{\beta_{ 2}}^{2}\over\beta_{1}}},$ $$

we obtain

$$ ${e^{-2\beta_{1}|x|^{2}}}I_{i}\leq 2^{n}e^{\alpha_{1}}\left({2\over\sqrt{1- \alpha_{2}}}\right)^{n+3}e^{-{K_{1}^{2}|x|^{2}\over 4}}\leq e^{-{{\beta_{2}}^{ 2}\over\beta_{1}}}(1+4\beta_{1})^{-{n\over 2}}.$ $$

By (49),

$$ $\mathrm{I_{i}}\leq\bar{p}(t,x).$ $$

Hence (50) holds. As an immediate conclusion, we have that

$$ $|\nabla\bar{q}|={|\nabla\bar{p}|\over\bar{p}}\leq\beta_{1}|x|+\beta_{2}+2\beta _{1}\sqrt{2}K|x|<3\beta_{1}K|x|+\beta_{2}.$ $$

Secondly, to prove (22) and (23), we first assume that $g(0)=0$ and $\nabla g(0)=0$, which will be removed at the end. Then

$$ $|\nabla g(x)|\leq L|x|.$ $$

Next we first verify (22). Note that

$$ $D^{2}\bar{q}=A-B+D\bar{q}\otimes D\bar{q}$ (51) $$

Here

$$ $A={1\over\bar{p}}{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2} }}hD^{2}g(x-\sqrt{2t}y)\,dy$ $$

and

$$ $B={1\over\bar{p}}{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2} }}hDg\otimes Dg(x-\sqrt{2t}y)\,dy.$ $$

Here $(u\otimes u)_{ij}=u_{i}u_{j}$ is the outer product. By Cauchy inequality, $B\geq D\bar{q}\otimes D\bar{q}$. Hence

$$ $D^{2}\bar{q}\leq\mathrm{A}\leq LI_{n}.$ (52) $$

For the other direction,

$$ $D^{2}\bar{q}\geq{\mathrm{A}}-{\mathrm{B}}\geq-LI_{n}-\mathrm{B}.$ $$

So we just need to estimate the term B. Note that

$$ $\begin{array}[]{ll}\left|\left|{{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{ n}}e^{-{|y|^{2}}}hDg\otimes Dg(x-\sqrt{2t}y)\,dy}\right|\right|_{2}&\leq{L^{2} \over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}(|x|+\sqrt{2}|y|)^{2}e^{-{|y|^{ 2}}}h\,dy\[14.22636pt] &\leq{2L^{2}\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}(|x|^{2}+{2}|y|^{2} )e^{-{|y|^{2}}}h\,dy\[14.22636pt] &\leq 2L^{2}\left(|x|^{2}\bar{p}(t,x)+{2\over{\pi}^{n\over 2}}\int_{{\mathbb{R }}^{n}}|y|^{2}e^{-{|y|^{2}}}h\,dy\right).\end{array}$ $$

Thanks to (50) for $\beta_{1}=L$, $\beta_{2}=0$ and $K=K_{0}$, we have that

$$ $||\mathrm{B}||_{2}\leq 2L^{2}(|x|^{2}+4K_{0}^{2}|x|^{2})<10L^{2}K_{0}^{2}|x|^{ 2}.$ $$

Hence

$$ $||D^{2}\bar{q}||_{2}\leq L+||B||_{2}<L+10L^{2}K_{0}^{2}|x|^{2}$ $$

Then let us verify (23). Note that

$$ $\nabla\bar{q}_{t}(t,x)=C+D-\bar{q}_{t}\nabla\bar{q}.$ $$

with

$$ $\begin{array}[]{ll}|C|&=\left|{1\over\sqrt{2t}}{1\over\bar{p}}{1\over{(\pi)}^{ n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}hD^{2}g(x-\sqrt{2t}y)\cdot y\, dy\right|\leq{L\over\sqrt{2t}}{1\over\bar{p}}{1\over{(\pi)}^{n\over 2}}\int_{{ \mathbb{R}}^{n}}|y|e^{-{|y|^{2}}}h\,dy\[14.22636pt] &\leq{2L\over\sqrt{2t}}K|x|<{2L\over\sqrt{t}}K|x|.\end{array}$ $$

Also,

$$ $\begin{array}[]{ll}|D|=&\left|{1\over\sqrt{2t}}{1\over\bar{p}}{1\over{(\pi)}^{ n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}h\nabla g\otimes\nabla g(x- \sqrt{2t}y)\cdot y\,dy\right|\[14.22636pt] &\leq{1\over\sqrt{2t}}{2L^{2}\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}(| x|^{2}|y|+{2}|y|^{3})e^{-{|y|^{2}}}h\,dy\[14.22636pt] &<{1\over\sqrt{t}}{2L^{2}|x|^{2}\over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n} }|y|e^{-{|y|^{2}}}h\,dy+{1\over\sqrt{t}}{4L^{2}\over{(\pi)}^{n\over 2}}\int_{{ \mathbb{R}}^{n}}|y|^{3}e^{-{|y|^{2}}}h\,dy\[14.22636pt] &\leq{2L^{2}\over\sqrt{t}}(2K|x|^{3}+4K^{3}|x|^{3})\[14.22636pt] &<{12L^{2}\over\sqrt{t}}K^{3}|x|^{3}.\end{array}$ $$

Finally,

$$ $\begin{array}[]{ll}|\bar{q}_{t}|&=\left|{1\over\sqrt{2t}}{1\over\bar{p}}{1 \over{(\pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}h\nabla g(x-\sqrt {2t}y)\cdot y\,dy\right|\[14.22636pt] &\leq{L\over\sqrt{2t}}{1\over\bar{p}}{1\over{(\pi)}^{n\over 2}}\int_{{\mathbb{ R}}^{n}}(|x||y|+\sqrt{2}|y|^{2})e^{-{|y|^{2}}}h\,dy\[14.22636pt] &\leq{L\over\sqrt{2t}}(2K|x|^{2}+2\sqrt{2}K^{2}|x|^{2})\leq{6L\over\sqrt{t}}K^ {2}|x|^{2}.\end{array}$ $$

Also,

$$ $|\nabla\bar{q}|\leq 2L(1+\sqrt{2})K|x|<6LK|x|.$ $$

Hence

$$ $|\bar{q}_{t}\nabla q|\leq{36L^{2}\over\sqrt{t}}K^{3}|x|^{3}.$ $$

So

$$ $\begin{array}[]{ll}|\nabla\bar{q}_{t}(t,x)|&\leq{2L\over\sqrt{t}}K|x|+{1\over \sqrt{t}}12L^{2}K^{3}|x|^{3}+{36L^{2}\over\sqrt{t}}K^{3}|x|^{3}\[14.22636pt] &\leq{50L^{2}\over\sqrt{t}}K^{3}|x|^{3}.\end{array}$ $$

At last, for a general $g$, we consider

$$ $g_{0}(x)=g(x)-g(0)-\nabla g(0)\cdot x.$ $$

and let $\bar{q}_{0}(t,x)$ be the corresponding score function. Then we have

$$ $g_{0}(0)=0,\quad\nabla g_{0}(0)=0\quad\mathrm{and}\quad\bar{q}_{0}(t,x)=\bar{q }(t,x+\nabla g(0)).$ $$

Hence (22) and (23) hold for general cases.
$\square$

###### Remark C.2

In the proof of (23), we may also bound $\bar{q}_{t}$ using the equation

$$ $\bar{q}_{t}={\frac{1}{2}}(\Delta\bar{q}-|\nabla\bar{q}|^{2})$ $$

together with bounds for $\Delta\bar{q}$ and $|\nabla\bar{q}|^{2}$. Note $\nabla g(x-\sqrt{t}y)=\nabla g(x)+r_{t}$ for $|r_{t}|\leq L\sqrt{t}|y|$. Then the term $D$ in the proof

$$ $\displaystyle\begin{array}[]{ll}D&={1\over\sqrt{2t}}{1\over\bar{p}}{1\over{( \pi)}^{n\over 2}}\int_{{\mathbb{R}}^{n}}e^{-{|y|^{2}}}h\nabla g\otimes\nabla g (x-\sqrt{2t}y)\cdot y\,dy\\ &=\bar{q}_{t}\nabla g(x)+O(L^{2}K^{3}|x|^{3}).\end{array}$ $$

This will lead to a bound of $\nabla\bar{q}_{t}$ by replacing $L^{2}\over\sqrt{t}$ in (23) by $O(nL^{3})$. The details are left to interested readers as an exercise.

Below are other simple situations where we can obtain a global uniform bound of the Hessian, which follows immediately from (48) and (51).

###### Theorem C.3

Let $L_{1}$ and $L_{2}$ be two positive constants such that

$$ $|\nabla g|\leq L_{1}\quad\mathrm{and}\quad||D^{2}g||_{2}\leq L_{2}.$ $$

Then

$$ $|\nabla\bar{q}(t,x)|\leq L_{1}\quad\mathrm{and}\quad-(L_{2}+L_{1}^{2})I_{n} \leq D^{2}q(t,x)\leq L_{2}I_{n}$ $$

###### Remark C.2

###### Theorem C.3

### C.4 Proof of Theorem 3.10

We first prove (1). Note that $D\bar{q}=-{D\bar{p}\over\bar{p}}$ and

$$ $\begin{array}[]{ll}\bar{p}(t,x)&={1\over{(2\pi t)}^{n\over 2}}\int_{{\mathbb{R }}^{n}}e^{-{|x-y|^{2}\over 2t}}d\pi_{0}(y)\\ &={1\over{(2\pi t)}^{n\over 2}}\int_{D_{0}}e^{-{|x-y|^{2}\over 2t}}d\pi_{0}(y) .\end{array}$ $$

Then

$$ $\nabla\bar{p}(t,x)=-{1\over{(2\pi t)}^{n\over 2}}\int_{D_{0}}{(x-y)\over t}e^{ -{|x-y|^{2}\over 2t}}d\pi_{0}(y).$ $$

Since $D_{0}\subset{\overline{B}_{M}(0)}$

$$ $|\nabla\bar{p}(t,x)|\leq{|x|+M\over t}\bar{p}(t,x).$ $$

Next we prove (25). Note that

$$ $-D^{2}\bar{q}(t,x)={D^{2}\bar{p}\over\bar{p}}-{\nabla\bar{p}\otimes\nabla\bar{ p}\over\bar{p}^{2}}=-{\delta_{ij}\over t}+{1\over t^{2}}{A-B\over\hat{p}^{2}(t ,x)}.$ $$

Here $\hat{p}(t,x)=\int_{D_{0}}e^{-{|x-y|^{2}\over 2t}}d\pi_{0}(y)$,

$$ $A_{ij}=\hat{p}(t,x)\int_{D_{0}}(x_{i}-y_{i})(x_{j}-y_{j})e^{-|x-y|^{2}\over 2t }d\pi_{0}(y)$ $$

and

$$ $\begin{array}[]{ll}B_{ij}&=\int_{D_{0}}(x_{i}-y_{i})e^{-|x-y|^{2}\over 2t}d\pi _{0}(y)\cdot\int_{D_{0}}(x_{j}-y_{j})e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\\ &=\hat{p}^{2}(t,x)x_{i}x_{j}-\hat{p}(t,x)\int_{D_{0}}(x_{i}y_{j}+x_{j}y_{i})e^ {-|x-y|^{2}\over 2t}d\pi_{0}(y)+\\ &\quad+\int_{D_{0}}y_{i}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\cdot\int_{D_{0}}y_{j }e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\end{array}$ $$

Hence $(A-B)_{ij}$

$$ $\hat{p}(t,x)\int_{D_{0}}y_{i}y_{j}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)-\int_{D_{0 }}y_{i}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\cdot\int_{D_{0}}y_{j}e^{-|x-y|^{2} \over 2t}d\pi_{0}(y)$ (53) $$

So it is easy to see that

$$ $||A-B||_{2}\leq M^{2}\hat{p}^{2}(t,x).$ $$

Thus (25) holds. $\square$

### C.5 Proof of Theorem 3.11

Since $\pi_{0}$ is comparable to the uniform distribution, there exists a constant $C$ such that for any measurable subset $U\subset D_{0}$

$$ ${1\over C}\mathcal{H}_{d}(U)\leq\pi_{0}(S)\leq C\mathcal{H}_{d}(U)$ (54) $$

Here $\mathcal{H}_{d}(\cdot)$ is the $d$-dimensional Hausdorff measure. Hereafter, we write

- (i)
$\partial D_{0}$: the $d-1$ dimensional boundary of $D_{0}$. Moreover, for $y\in D_{0}$;
- (ii)
$T_{y}(D_{0})\subset{\mathbb{R}}^{n}$: the $d$-dimensional tangent space of $D_{0}$ at $y$;
- (iii)
$N_{y}(D_{0})\subset{\mathbb{R}}^{n}$: the $n-d$ dimensional orthogonal complement of $T_{y}(D_{0})$;
- (iv)
$T^{\prime}_{y}(D_{0})\subset{\mathbb{R}}^{n}$: the $d-1$-dimensional tangent space of $\partial D_{0}$ at $y\in\partial D_{0}$. Note that $T^{\prime}_{y}(D_{0})$ is a subspace of $T_{y}(D_{0})$;
- (v)
$N^{\prime}_{y}(D_{0})\subset{\mathbb{R}}^{n}$: the $n+1-d$ dimensional orthogonal complement of $T^{\prime}_{y}(D_{0})$. $N_{y}(D_{0})$ is a subspace of $N^{\prime}_{y}(D_{0})$

Let

$$ $S=\{x\in{\mathbb{R}}^{n}|\ \text{there exists a unique $y_{x}\in D_{0}$ such that $|x-y_{x}|=d(x,D_{0})$}\}.$ $$

Then ${\mathbb{R}}^{n}\backslash S$ has zero measure since $d(x,D_{0})$ is differentiable almost everywhere.

Write

$$ $S_{1}=\{x\in S|\ y_{x}\in D_{0}\backslash\partial D_{0}\}\quad\mathrm{and} \quad S_{2}=\{x\in S|\ y_{x}\in\partial D_{0}\},$ $$

$$ $W_{1}=\left\{x\in S_{1}|\ \liminf_{y\in D_{0}\to y_{x}}{|x-y|^{2}-|x-y_{x}|^{2 }\over|y-y_{x}|^{2}}>0\right\},$ $$

$$ $W_{2}=\left\{x\in S_{2}|\ x-y_{x}\in N_{y_{x}}(D_{0})\quad\mathrm{and}\quad \liminf_{y\in D_{0}\to y_{x}}{|x-y|^{2}-|x-y_{x}|^{2}\over|y-y_{x}|^{2}}>0 \right\},$ $$

and

$$ $W_{3}=\left\{x\in S_{2}|\ x-y_{x}\notin N_{y_{x}}(D_{0})\quad\mathrm{and}\quad \liminf_{y\in\partial D_{0}\to y_{x}}{|x-y|^{2}-|x-y_{x}|^{2}\over|y-y_{x}|^{2 }}>0\right\}.$ $$

Note that

$$ $\begin{array}[]{ll}\text{ if $y\in D_{0}\backslash\partial D_{0}$},\quad\text{ then $x-y_{x}\in N_{y_{x}}(D)$},\\ \text{ if $y\in\partial D_{0}$},\quad\text{then $x-y_{x}\in N^{\prime}_{y_{x}} (D)$}.\end{array}$ $$

Step 1: We show that $S\backslash\left(\cup_{i=1}^{3}W_{i}\right)=(S_{1}\backslash W_{1})\cup(S_{2}
\backslash W_{2})\cup(S_{2}\backslash W_{3})$ has zero measure. We will prove this for $S_{1}\backslash W_{1}$, The proofs for the other two are similar. Apparently, if $x\in S$, then for all $t\in(0,1)$, $y_{x_{t}}=y_{x}$ and $x_{t}\in W$ for $x_{t}=y_{x}+t(x-y_{x})$. Also,
for $y\in D_{0}\backslash\partial D_{0}$, let us write

$$ $\Gamma_{y}=\{x\in S_{1}\backslash W_{1}|\ y_{x}=y\}.$ $$

By compactness argument, it is easy to show that for given $x\in S$ and $r>0$, there exists a $r_{x}>0$, such that $y_{\tilde{x}}\in B_{r}(y_{x})$ for any $\tilde{x}\in B_{r_{x}}(x)\cap S$ . Hence to prove that $S_{1}\backslash W$ has zero measure, it suffices to show that for any $y_{0}\in D_{0}$, if $\Gamma_{y_{0}}$ is not empty, then there exists $r_{0}>0$ such that

$$ $\cup_{y\in B_{r_{0}}(y_{0})\cap D_{0}}\Gamma_{y}\subset\{y+t(y,v)|\ y\in B_{r_ {0}}(y_{0}),\ v\in N_{y}(D_{0})\ \mathrm{and}\ |v|=1\}$ (55) $$

for a locally Lipschitz continuous function

$$ $t(y,v):B_{r_{0}}(y_{0})\times N_{y}(D_{0})\to(0,\infty).$ $$

By suitable translation and rotation, we may assume $y_{0}=0$ and in a neighbourhood $V$ of 0 0,

$$ $D_{0}\cap V=V\cap\{(y^{\prime},F(y^{\prime}))|\ y^{\prime}\in{\mathbb{R}}^{d}\},$ (56) $$

where $F=(F^{(d+1)},F^{(d+2)},...,F^{(n)}):{\mathbb{R}}^{d}\to{\mathbb{R}}^{n-d}$ is a smooth map satisfying $\nabla F(0)=0$ and $F(0)=0$. Choose $x\in\Gamma_{y_{0}}$. Let $v={x\over|x|}\in N_{y}(D_{0})$ and $t>0$, let

$$ $H_{tv}(y^{\prime})=|tv-(y^{\prime},F(y^{\prime}))|^{2}.$ $$

Since $x\in W_{1}$, $D^{2}H_{tv}(0)$ cannot be a positive definite matrix for $t=|x|$. Meanwhile, for $0<t<|x|$, $y_{tx}=y_{0}$ and $tx\in W_{1}$, which implies that $D^{2}H_{tv}(0)$ is positive definite for $t\in[0,|x|)$. Hence $t=|x|$ is the first moment such that $D_{tv}^{2}H(0)$ has a zero eigenvalue,
which is equivalent to saying that

$$ the largest eigenvalue of the $d\times d$ matrix $\sum_{k=d+1}^{n}tv_{k}F^{(k)}_{y^{\prime}_{i}y^{\prime}_{j}}(0)$ is 1. $$

Therefore, for $y\in B_{r}(y_{0})$ and $v\in T_{y}(D_{0})$ with $|v|=1$, if the largest eigenvalue $\lambda(y,v)$ of the matrix $\sum_{k=d+1}^{n}v_{k}F^{(k)}_{y^{\prime}_{i}y^{\prime}_{j}}(y)$ is positive, we set

$$ $t(y,v)={1\over\lambda(y,v)}.$ $$

Then (55) holds.

Step 2: We will verify that if $x\in\cup_{i=1}^{3}W_{i}$, then

$$ $|D^{2}\bar{q}(t,x)|\leq{C_{x}\over t}\quad\text{for all $t\in(0,1]$}.$ $$

Case 1: Assume that $x\in W_{1}$. Without loss of generality, we may assume $y_{x}=0$ and use the representation as (56). Choose $r>0$ such that

(i)$V_{r}=\{(y^{\prime},F(y^{\prime}))|\ |y^{\prime}|<r\}\subset D_{0}$;

(ii) Then there exists $\alpha_{x},\beta_{x}>0$ such that for $y\in V_{r}$,

$$ $\alpha_{x}|y-y_{x}|^{2}\geq|x-y|^{2}-|x-y_{x}|^{2}\geq\beta_{x}|y-y_{x}|^{2} \quad\text{for $y\in D_{0}$}.$ (57) $$

For $k\geq 1$ and $k\sqrt{t}\leq r$, write

$$ $V_{t,k}=\left\{(y^{\prime},F(y^{\prime}))|\ |y^{\prime}|<k\sqrt{t}\right\}$ $$

Thanks to the left upper bound in (57),

$$ $\int_{D_{0}}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\geq\int_{V_{t,1}}e^{-|x-y|^{2} \over 2t}\,d\pi_{0}\geq O\left(t^{d\over 2}e^{-{|x-y_{x}|^{2}\over 2t}}\right).$ $$

Recall that $y_{x}=0$. To see the dependence on $y_{x}$, we keep $y_{x}$ in the computations below instead of replacing it by 0 0. Note

$$ $\int_{D_{0}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\leq\int_{V_{r}}|y-y_ {x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}+Ce^{-{{|x-y_{x}|^{2}+\delta_{r}}\over 2 t}}$ $$

for some $\delta_{r}>0$.

Also,

$$ $\begin{array}[]{ll}&\int_{V_{r}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}= \sum_{k=0}^{\infty}\int_{\{y\in V_{t,k+1}\backslash V_{t,k}\}}|y-y_{x}|^{2}e^{ -|x-y|^{2}\over 2t}\,d\pi_{0}\\ &\leq Ct\cdot t^{d\over 2}e^{-{|x-y_{x}|^{2}\over 2t}}\sum_{k=1}^{\infty}(k+1) ^{2}e^{-k}=O\left(t^{d\over 2}e^{-{|x-y_{x}|^{2}\over 2t}}\right)t.\end{array}$ $$

Hence

$$ ${\int_{D_{0}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}}\leq Ct$ (58) $$

Recall that

$$ $\hat{p}(t,x)={\int_{D_{0}}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)}=(\sqrt{2\pi t})^{ n}\bar{p}(t,x).$ $$

Then for $1\leq i,j\leq n$ and $y_{x}=(a_{1},a_{2},...,a_{n})$,

$$ $\begin{array}[]{ll}&\left|{\int_{D_{0}}y_{i}y_{j}e^{-|x-y|^{2}\over 2t}d\pi_{0 }(y)\over\hat{p}(x,t)}-{\int_{D_{0}}y_{i}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y) \over\hat{p}(x,t)}\cdot{\int_{D_{0}}y_{j}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y) \over\hat{p}(x,t)}\right|=\\ &\left|{{\int_{D_{0}}(y_{i}-a_{i})(y_{j}-a_{j})e^{-|x-y|^{2}\over 2t}d\pi_{0}( y)\over\hat{p}(x,t)}}-{\int_{D_{0}}(y_{i}-a_{i})e^{-|x-y|^{2}\over 2t}d\pi_{0} (y)\over\hat{p}(x,t)}\cdot{\int_{D_{0}}(y_{j}-a_{j})e^{-|x-y|^{2}\over 2t}d\pi _{0}(y)\over\hat{p}(x,t)}\right|\\ &\leq Ct.\end{array}$ $$

The last equality follows from (58) and the Cauchy inequality. Therefore, (53) leads to

$$ $|D^{2}\bar{q}(x,t)|\leq{C_{x}\over t}.$ $$

Case 2: $x\in W_{2}$. The proof is similar to Case 1.

Case 3: $x\in W_{3}$. By suitable translation and rotation, we may assume $y_{x}=0$ and in a neighborhood of $0\in{\mathbb{R}}^{d}$,

$$ $D_{0}\cap V=\tilde{V}_{r}=\{(y^{\prime},F(y^{\prime}))|\ y^{\prime}=(y^{\prime }_{1},...,y^{\prime}_{d})\in\Omega_{f,r}\},$ $$

where $F:{\mathbb{R}}^{d}\to{\mathbb{R}}^{n-d}$ is smooth map satisfying that $\nabla F(0)=0$. Also,

$$ $\Omega_{f,r}=\{z=(z^{\prime},z_{d})|\ z^{\prime}=(z_{1},z_{2},..,z_{d-1})\in{ \mathbb{R}}^{d-1},\ |z^{\prime}|<r;\;z_{d}\geq f(z_{1},z_{2},..,z_{d-1})\}.$ $$

for a smooth function $f:{\mathbb{R}}^{d-1}\to{\mathbb{R}}$ subject to $\nabla f(0)=0$. Then

$$ $T_{0}(D_{0})=\{(v,0,...,0)\in{\mathbb{R}}^{n}\ |\ v\in{\mathbb{R}}^{d}\}$ $$

and the $d-1$ dimensional tangent plane to $\partial D_{0}$ at $y_{x}=0$ is

$$ $\partial T_{0}(D_{0})=\{(v^{\prime},0,0,...,0)\in{\mathbb{R}}^{n}\ |\ v^{ \prime}\in{\mathbb{R}}^{d-1}\}.$ $$

Thus

$$ $x=x-y_{x}=(\underbrace{0,...0}_{d-1},\theta_{x},z_{x})$ $$

for some $\theta_{x}>0$ and $z_{x}\in{\mathbb{R}}^{n-d}$. To see the dependence on $y_{x}$, as in Case 1, we keep $y_{x}$ in the computations below instead of replacing it by 0 0.

Then for $y^{\prime\prime}\in{\mathbb{R}}^{d-1}$ and $y=(y^{\prime\prime},y_{d},F(y^{\prime\prime},y_{d}))\in D_{0}$,

$$ $|x-y|^{2}-|x-y_{x}|^{2}=2\theta_{x}(y_{d}-f(y^{\prime}))+O(|y-y_{x}|^{2}).$ (59) $$

Write

$$ $H(y)=|x-y|^{2}-|x-y_{x}|^{2}=H(y^{\prime\prime},y_{d},F(y^{\prime\prime},y_{d} )).$ $$

Since $x\in W_{3}$,

$$ $H(y^{\prime\prime},f(y^{\prime\prime}),F(y^{\prime\prime},f(y^{\prime\prime})) )\geq\delta_{x}|y^{\prime\prime}|^{2}.$ $$

Therefore, there exists $r>0$ and $M>0$ such that

$$ $|x-y|^{2}-|x-y_{x}|^{2}\geq{\theta_{x}\over M}(y_{d}-f(y^{\prime}))+\delta_{x} |y-y_{x}|^{2}\quad\text{for all $y\in\tilde{V}_{r}$}.$ (60) $$

Write

$$ $R_{t,k}=\{(y^{\prime},y_{d},F(y^{\prime},y_{d}))\in\Omega|\ |y^{\prime}|\leq k \sqrt{t}\quad\mathrm{and}\quad 0\leq y_{d}-f(y^{\prime})\leq kt\}.$ $$

Thanks to (59),

$$ $\hat{p}(t,x)=\int_{D_{0}}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\geq\int_{D_{0}\cap R _{t,1}}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\geq O\left(t^{d+1\over 2}e^{-{|x-y_{x} |^{2}\over 2t}}\right).$ $$

Note that

$$ $\int_{D_{0}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\leq\int_{\tilde{V}_{ r}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}+Ce^{-{|x-y_{x}|^{2}+\delta_{r }\over 2t}}.$ $$

Also,

$$ $\begin{array}[]{ll}&\int_{\tilde{V}_{r}}|y-y_{x}|^{2}e^{-|x-y|^{2}\over 2t}\,d \pi_{0}=\sum_{k=0}^{\infty}\int_{D_{0}\cap(R_{t,k+1}\backslash R_{t,k})}|y-y_{ x}|^{2}e^{-|x-y|^{2}\over 2t}\,d\pi_{0}\\ &\leq Ct\cdot t^{d+1\over 2}e^{-{|x-y_{x}|^{2}\over 2t}}\sum_{k=1}^{\infty}(k+ 1)^{2}e^{-k}=tO\left(t^{d+1\over 2}e^{-{|x-y_{x}|^{2}\over 2t}}\right).\end{array}$ $$

Hence

$$ ${\int_{D_{0}}|y-\bar{y}_{x}|^{2}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}} \leq Ct$ $$

Then by the same argument in the end of Case 1, we deduce that

$$ $||D^{2}\bar{q}(x,t)||_{2}\leq{C_{x}\over t}.$ $$

Finally, if $D_{0}$ is convex, then it is clear that $S={\mathbb{R}}^{n}$ and $W_{1}=S_{1}$ and $W_{2}\cup W_{3}=S_{2}$. Hence the $O({1\over t})$ bound holds for all $x\in{\mathbb{R}}^{n}$.
$\square$

### C.6 Proof of Example 3.12

Proof: For given $x\in{\mathbb{R}}^{2}$, denote by $\bar{y}_{t}$ the weighted center of mass:

$$ $\bar{y}_{t}={\int_{D_{0}}ye^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}}$ $$

Note that as $t\to 0$, the measure $e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}$ will concentrate on $\{y\in D_{0}|\ |y-x|=d(x,D_{0})\}$. Thus,

$$ $\lim_{t\to 0}d(\bar{y}_{t},\Gamma_{x})=0,$ $$

where $\Gamma_{x}$ is the convex hull of $\{y\in D_{0}|\ |y-x|=d(x,D_{0})\}$. According to the computation in the proof of (25), we have that

$$ $\displaystyle-\Delta\bar{q}=$ $\displaystyle-{n\over t}+{\hat{p}\int_{D_{0}}|y|^{2}e^{-|x-y|^{2}\over 2t}d\pi _{0}(y)-\left|\int_{D_{0}}ye^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\right|^{2}\over t ^{2}\hat{p}^{2}}$ (61) $\displaystyle=$ $\displaystyle-\Delta\bar{q}=-{n\over t}+{\int_{D_{0}}|y-\bar{y}_{t}|^{2}e^{-|x -y|^{2}\over 2t}d\pi_{0}(y)\over t^{2}\hat{p}},$ (62) $$

where the second term is like a variance.
If $x=(\theta,0)$ for some $\theta>0$, there are two points $y_{1}$ and $y_{2}$ such that

$$ $|x-y_{1}|=|x-y_{2}|=d(x,D_{0}).$ $$

Due to the symmetry, we must have that

$$ ${e^{-|x-y|^{2}\over 2t}d\pi_{0}(y)\over\hat{p}}\to{\frac{1}{2}}\delta_{y_{1}}+ {\frac{1}{2}}\delta_{y_{2}}\quad\mathrm{and}\quad\lim_{t\to+\infty}y_{t}={y_{1 }+y_{2}\over 2}.$ $$

Accordingly,

$$ $\lim_{t\to 0}{\int_{D_{0}}|y-\bar{y}_{t}|^{2}e^{-|x-y|^{2}\over 2t}d\pi_{0}(y) \over\hat{p}}={|y_{1}-y_{2}|^{2}\over 4},$ $$

leading to

$$ $-\Delta\bar{q}(x,t)\geq{C_{x}\over t^{2}}\quad\text{for $t\in(0,1]$.}\quad \quad\quad\mbox{$\square$}\newline$ $$

### C.7 Proof of Theorem 4.1

###### Lemma C.4

Given $T>0$, suppose that $F=F(t,x)\in C([0,T]\times{\mathbb{R}}^{n},{\mathbb{R}}^{n})$ satisfies that $F$ is locally Lipschitz continuous in $x$ variable, i.e., for any $M>0$, there exists a constant $L_{M}$ such that

$$ $|F(t,x)-F(t,y)|\leq L_{M}|x-y|\quad\text{for $x,y\in B_{M}(0)$ and $t\in[0,T]$}$ $$

and

$$ $|F(t,x)|\leq C(|x|+1).\quad\text{for $(t,x)\in[0,T]\times{\mathbb{R}}^{n}$}.$ (63) $$

for a positive constant C. Then for any $x_{0}\in{\mathbb{R}}^{n}$, the following equation has a unique solution

$$ $\begin{cases}\dot{X}(t)=F(t,X(t))\quad\text{$t\in[0,T]$}\\ X(0)=x_{0}.\end{cases}$ $$

Proof: The uniqueness follows from standard ODE theory. We just need to establish the global existence. Let $w(t)=|X(t)|^{2}$. Then

$$ $\dot{w}(t)\leq C_{1}w(t)+C_{2}$ $$

for two positive constants $C_{1}$ and $C_{2}$ depending only on $C$. Hence for all $t\geq 0$,

$$ $e^{-C_{1}t}w(t)\leq|x_{0}|^{2}+{C_{2}\over C_{1}}\left(1-e^{-C_{1}t}\right).$ $$

Hence the solution can be extended to $T$. $\square$

###### Lemma C.4

#### Proof of Theorem 4.1

It suffices to notice that for each fixed sample $\omega$, $Y(t)=Y(t,\omega)=X_{t}(\omega)-W_{t}(\omega)$ just satisfies the regular ODE for any fixed sample

$$ $\begin{cases}dY(t)=F(Y+W(t),t)dt\quad\text{$t\in[0,\infty)$}\\ Y(0)=x_{0}.\end{cases}$ $$

Hence the Corollary follows from Theorem C.4 and the well-known fact that $W_{t}(\omega)\in C([0,T],{\mathbb{R}}^{n})$ for a.e. $\omega$.
$\square$

### C.8 Proof of Theorem 4.4

The key ingredient is the following estimates on truncation error.

###### Theorem C.5

Under the same assumption as Corollary 3.7, we have that for fixed $T\leq 1$ and $t_{k}={kT\over N}$,

$$ $\sum_{i=1}^{N}\int_{t_{k-1}}^{t_{k}}\mathbb{E}||\nabla\bar{q}(t_{k},x(t_{k}))- \nabla\bar{q}(t,x(t))||^{2}\,dt\leq{CL^{6}Tn(n\log n)^{2}\over N}$ (64) $$

Here $C$ is a constant independent of $n$ and $L$.

Proof: It suffices to show that for $s>t\in[0,T]$,

$$ $\mathbb{E}||\nabla\bar{q}(s,x(s))-\nabla\bar{q}(t,x(t))||^{2}\leq CL^{6}(s-t)n ^{3}\log n$ $$

According to Lemma C.6 in [], it suffices to show that

$$ $\mathbb{E}||\nabla\bar{q}(t,x(t)+z)-\nabla\bar{q}(t,x(t))||^{2}\leq Cn^{2}(s-t).$ (65) $$

Here $z\thicksim\mathcal{N}(0,C(s-t))$.

Owing to Theorem 3.6 in our paper,

$$ $||D^{2}\bar{q}(t,x)||_{2}\leq C(|x|^{2}+n\log n).$ $$

See (16) for the definition of the spectral norm $||\cdot||_{2}$ of $n\times n$ matrix. Then

$$ $||\nabla\bar{q}(t,x(t)+z)-\nabla\bar{q}(t,x(t))||^{2}\leq C(1+|x(t)|^{4}+|z|^{ 4}+(n\log n)^{2})|z|^{2}.$ $$

Note that $\mathbb{E}(z^{2})\leq Cn(s-t)$ and $\mathbb{E}(z^{4})\leq Cn^{2}(s-t)^{2}$. Moreover, by Cauchy inequality

$$ $\mathbb{E}(|x(t)|^{4}z^{2})\leq\sqrt{(\mathbb{E}(x^{8}(t))\mathbb{E}(z^{4})} \leq Cn(n\log n)^{2}(s-t).$ $$

The last inequality is due to $\mathbb{E}_{p(t,x)}(x(t)^{8})\leq C(n\log n)^{4}$ from Remark 3.8. Hence (65)holds. $\square$

###### Remark C.6

In the proof of Theorem C.5, instead of using Lemma C.6 in [], we may also use (23) from Theorem 3.6 to bound the difference between two times and so:

$$ $\mathbb{E}||\nabla\bar{q}(s,x(t))-\nabla\bar{q}(t,x(t))||^{2}\leq C(s-t)^{2}n^ {4}(\log n)^{3}.$ $$

This leads to an extra term ${Cn^{4}(\log n)^{3}\over N^{2}}$ on the right hand side of (64). The proof is similar. Note that when $N=O(n^{2})$, ${n^{4}(\log n)^{3}\over N^{2}}\preceq{n^{3}(\log n)^{2}\over N}$.

###### Remark C.7

We are aware of the difference of $\nabla\log p$ and $\nabla\bar{q}$ due to the translation (11), while our Lipschitz estimate is uniform in time, hence similar results of Theorem C.5 hold for $\nabla\log p$.

###### Theorem C.5

###### Remark C.6

###### Remark C.7

### C.9 Convergence bounds under compact support manifold assumption

###### Theorem C.8

Assume that $\mathrm{supp}(p_{0})=D_{0}\subset\overline{B_{M}(0)}$ and the density is smoothly defined on $D_{0}$. Consider early stopping, so let $\delta>0$, and let $\hat{q}_{T-\delta}$ be the distribution generated by the uniform discretization scheme of the exponential integrator (4), with an approximated score satisfying the Assumption 2.1.
If $L_{1}>0$, then

$$ $\displaystyle\mathrm{KL}(p_{\delta}\|\hat{q}_{T-\delta})\lesssim(M_{2}+d)e^{-T }+T\epsilon_{0}^{2}+\frac{dT^{2}L^{2}_{\delta}}{N},$ $$

where $L_{\delta}=1+\frac{1}{\delta}+\frac{M^{2}}{\delta^{2}}$.

Proof:
The $L_{\delta}$ is computed from Theorem 3.10. Then Proposition A.3 applies and yields the proof. $\square$

###### Theorem C.8

## Appendix D Broader Impact

Diffusion model is one of the most influential generative models in the era of AI. Our theory provides theoretical guarantee of diffusion models with minimal assumption on data distribution. In addition, we provide a theoretical characterization of the singular behaviour related to the manifold hypothesis. This provides insight for model parameterization in practical implementations.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: All the questions have been addressed in the *Weaknesses* section.
soundness: 4
strengths: The major strengths of this paper are:
- Establishing crucial bounds on the Hessian of the score function, which significantly enhance existing bounds on the KL divergence from *Chen et al. (2023)*. These bounds are vital for assessing the fidelity of generated distributions.
- Carefully tracking the dependence on dimensionality throughout the analysis, a critical aspect for score-based generative models (SGMs) given their frequent application to high-dimensional data distributions.
- Characterizing the explosion of the score function as $t$ approaches $0$. Although this instability near zero has been commonly observed in practice, the authors quantify the explosion’s magnitude through PDE techniques, opening the door to improve model stability in high-dimensional generative tasks.
- Describing the case of compactly supported data distributions, shedding light on the theoretical limits of score-based generative models (SGMs). This perspective allows for a deeper understanding of how SGMs behave when data is restricted within a bounded region, which has practical relevance given that real-world data often exhibit such constraints.
summary: This paper investigates the mathematical framework behind score-based generative models (SGMs). The authors' primary focus is to establish conditions under which these models are well-posed and convergent.

Key points from the paper include:
1. **Well-Posedness of SGMs:** The authors prove that SGMs are well-posed under global conditions. They introduce Lipschitz estimates that maintain stability and prevent the “blow-up” of the score function, which has been an issue in practical applications where score gradients become unstable.
2. **Convergence and Sharp Lipschitz Bounds:** They provide a Lipschitz bound for the score function, addressing both smooth and non-smooth cases. This bound helps establish global convergence of the backward process (used in sample generation), even for non-log-concave distributions. For non-smooth data distributions, the bound asymptotically scales as $1/t$, ensuring convergence on low-dimensional manifolds with compact support.
3. **Theoretical Contributions:** The authors introduce rigorous estimates for the Hessian and gradient of the score function, reinforcing the stability and convergence properties. This allows the generative models to better approximate complex data distributions.

The paper concludes with insights into how these findings could guide model implementation, such as early stopping strategies and parameter adjustments, to ensure stable and efficient generation in real-world applications.
weaknesses: Here are some weaknesses of the paper:
1. **Limited Comparison with Recent Literature:** The paper lacks sufficient engagement with recent literature on SGMs, particularly from a PDE perspective. A broader comparison with recent works is needed, especially regarding bounds on KL divergence, as several relevant contributions exist and should be cited.
2. **Redundancy in Subsection 2.2:** The computations in Subsection 2.2 are already present in the literature, as seen in *Conforti, Durmus, Gentiloni-Silveri (2023)*. While this content is valuable for readability, it might be better suited to an appendix section to streamline the main text.
3. **Unclear Novelty in Theorems 4.3 and 4.4:** The novelty of Theorems 4.3 and 4.4 is unclear in comparison to the results in *Chen et al. (2023)*. It would be beneficial to clarify how the bounds on the Hessians improve convergence rates relative to this paper, particularly in establishing any distinct contributions.
4. **Lack of Appendix Structure:** Adding a paragraph at the beginning of the appendix outlining its structure would improve navigation and clarity for readers.

Addressing these points would strengthen the paper, and I would consider raising the evaluation if these aspects are clarified and expanded.

---

confidence: 2
contribution: 2
initial_rating: 3
presentation: 2
questions: N/A
soundness: 2
strengths: The paper addresses an interesting question of trying to substantiate the Lipschitzness assumptions on score functions. The result on manifolds showing that 1/t^2 can be almost-everywhere improved to 1/t seems qualitatively new and interesting, and the example showing that this is tight provides a fairly complete story.
summary: Recent theoretical works on analyzing diffusion models require bounds on the Lipschitzness of the score function of the noised distribution at all time-steps. This work gives bounds on Lipschitzness under various conditions on the initial data distribution, and uses some of these bounds to instantiate prior guarantees for diffusion models.
- Theorems 3.1 and 3.4 show that if the initial data distribution is smooth, then the score of the noised distribution is Lipschitz up to some finite time T, and that this cannot be extended to all times
- Theorem 3.5 gives local Lipschitzness estimates of the score function under local assumptions
- Theorems 3.9/3.10 and Example 3.11 show that if the data distribution is supported on a manifold, then one gets Lipschitzness bounds of 1/t^2 at time t, and in fact almost everywhere one gets 1/t, and there is an example where the "almost everywhere" is necessary
- Theorem 4.3/4.4 instantiate diffusion model guarantees using Theorem 3.1 and Theorem 3.5 respectively
weaknesses: The paper could be written better, as far as comparison to prior work, clarity about which of the results are most "important", readable theorem statements, and discussion about the assumptions.
- Theorem 3.1, 3.9, and 4.1 seem as though they may be fairly standard / similar to prior results, but there is very little discussion about what is new and what is included merely for completeness.
- Theorem 4.4 says "with same assumption as Corollary 3.6", which in turn says "under the same assumption as in Theorem 3.5". The stated assumption is uninterpretable: is it reasonable? I can't tell. 
- The invocation of Theorem 3.5 in the proof (line 1320) provides no explanation as to how the complicated bound in Theorem 3.5 becomes this simple bound.

The most interesting result would be if the 1/t^2 -> 1/t improvement could be used to instantiate a better bound for diffusion models. However it appears that this is not done. The rest of the results in Section 4 just have lots of assumptions (e.g. the bound on T in Theorem 4.3) so it's not clear how applicable they are.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. It appears that the main assumption in this work is Assumption 2.3, which requires that the tail distribution is bounded by a Gaussian distribution. Could you provide some intuition behind this assumption? It seems potentially quite strong, as it may imply that the data is approximately Gaussian—perhaps even stronger than the strong dissipativity condition. Could you clarify why this is a reasonable assumption for real-world data distributions?

2. The statements in Section 3.3 appear somewhat inconsistent. In Theorem 3.9, it is assumed that the density is smoothly defined on $D_0$, which may be a low-dimensional manifold, while in the proof, the integral over $\mathbb{R}^n$ is equal to the integral over $D_0$. How is $d\pi_0(y)$ defined to ensure this equality holds? Is $p_0$ a density on $D_0$ or $\mathbb{R}^n$? I may have misunderstood, so please correct me if I'm mistaken.
soundness: 3
strengths: The paper introduces new sharp Lipschitz bounds for the score function, which enable global well-posedness and convergence without requiring separated time scales. This is particularly valuable for SGMs where score explosion has previously posed challenges.
summary: This paper addresses the global well-posedness and convergence of score-based generative models (SGMs) by providing sharp Lipschitz estimates for the score function under both smooth and non-smooth initial conditions. By developing optimal point-wise gradient and Hessian estimates, the authors establish conditions under which the backward diffusion process is well-posed up to time zero, effectively covering cases where previous approaches failed due to exploding scores or separated time regimes.
weaknesses: I did not identify any major weaknesses in this paper. I listed some typos below.

Typos: 
1. On the LHS of equation (3), I believe $p_t$ means $\partial_t p$. Using the notation $p_t$ will probably make the reader confused as $p_0$ denotes the initial data;
2. In line 108, it should be $Q_{t'}$ instead of $Q_t'$;
3. In line 127, the LHS should be $\hat{x}$ instead of $d\hat{x}$;
4. In line 218, it should be $\mathbb{R}^n$ instead of $\mathbb{R}^d$.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: (1) In the last paragraph of page 1, well-poseness should be well-posedness.

(2) In the last paragraph of Section 1, Planc should be Planck.

(3) On page 3, "In the analysis, we assume an $\epsilon^{2}$ bounds for this estimation'' I think you may replace $\epsilon$ by $\epsilon_{0}$ because this is what you used in Assumption 2.1.

(4) It would be nice to add some discussions about Assumption 2.3., how it is related to the assumptions in the previous theoretical work on score-based generative models.

(5) In the statement of Corollary 3.2, $p_{0}\in C^{2}(\mathbb{R}^{d})$ but you also use $\sup_{x\in\mathbb{R}^{n}}D^{2}\log p_{0}(x)$. I think in your case $d=n$? Similar issue appears in the statement of Theorem 4.3. and Theorem 4.4., where you have used both $d$ and $n$.

(6) In Theorem 4.3., in the upper bound for $\mathrm{KL}(p_{0}\Vert\hat{q}_{T})$, the first term is $(M_{2}+d)e^{-T}$. Can you remind the readers what is $M_{2}$ here (or where it is defined)? Also, for $H_{T}$, does it have an explicit formula (depending on $L_{0},L_{1},T$) and if so you should mention it. Also, I think it will be nice to spell out the dependence on $T$ because your upper bound is written in a way with an emphasis on the dependence on $T$, and if $H_{T}$ is implicit in $T$ it will make your bound less
valuable.

(7) In order to have $\mathrm{KL}(p_{0}\Vert\hat{q}_{T})$ to be small in  the upper bound for $\mathrm{KL}(p_{0}\Vert\hat{q}_{T})$ in Theorem 4.3., it seems you have to choose $T$ to be large, but on the other hand $T<-\log(1-\frac{1}{L_{0}+1})$, which seems
problematic to me. 

(8) If my understanding is correct, the main contribution of the paper is the Lipschitz estimate on the score function, rather than the convergence analysis, which follows from Chen et al. (2023) etc. from the literature. In that spirit, does that mean your Lipschitz estimate on the score function can also be used to obtain convergence analysis for probability flow ODEs by leveraging existing works on the convergence analysis of the probability flow ODEs in the literature?
soundness: 3
strengths: (1) Estimating the Lipschitz constant for the score-function using the idea of the PDE approach is novel and makes a nice contribution to the theory of score-based generative models.

(2) An example is provided to show the Lipschitz estimate is sharp in some sense.
summary: The paper focuses on providing the Lipschitz estimates for the score-function in the score-based generative models. It also provides a sharp example showing that the loss of Lipschitz bound of the score function as time gets large even with nice initial data distribution.  As a by-product, well-posedness and convergence are obtained by following existing literature.
weaknesses: (1) I think in the abstract, the paper overclaims that the results are obtained under minimal general assumptions of initial data because when you look at Theorem 3.1., Corollary 3.2. etc., some relatively strong assumptions are used.

(2) Well-posedness follows immediately from the estimate on the score function and Theorem 4.1. can be found in any textbook on stochastic calculus. Moreover, the convergence analysis follows immediately from existing literature (Chen et al. 2023). In that regard,
I think the main contributions of the paper are simply the Lipschitz estimates for the score function, and well-posedness and convergence analysis are not the main contributions of the paper.

Meta-review: The manuscript studies the score-based generative models. Compared with previous studies, the manuscript establishes sharper Lipschitz estimate of the score function, which in particular covers the non-smooth case when the data distribution might lie on lower-dimensional manifold. Overall, while it feels that the theoretical contribution towards understanding score-based diffusion models is marginal compared with the literature, the manuscript is solid and the authors have addressed all reviewers' concerns through the discussion phase. The reviewers unanimously recommends marginal above acceptance, and the meta-reviewer agrees with the assessment.

Decision: Accept (Poster)