<figure><img src="image_1.png" /></figure>

HAL
open science

Beyond the delta method

Antoine Lejay, Sara Mazzonetto

► To cite this version:

Antoine Lejay, Sara Mazzonetto. Beyond the delta method. 2025. hal-03738371v4

HAL Id: hal-03738371

https://inria.hal.science/hal-03738371v4

Preprint submitted on 25 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Beyond the delta method

Antoine Lejay* Sara Mazzonetto†

June 23, 2025

**Abstract**

We establish an asymptotic expansion for the maximum likelihood estimator (MLE), and more generally for estimators defined implicitly, in terms of the limiting behavior of the score function and its higher-order derivatives. The expansion is explicitly computable and yields a refined understanding of the non-asymptotic behavior of the renormalized MLE and its deviation from the limiting distribution. The results hold under general conditions, requiring only the convergence of the score and its derivatives, including in settings with non-Gaussian limits. Our approach builds on an asymptotic implicit function theorem, inspired by perturbative techniques.

**MSC(2020) Classification:** Primary 62F12; Secondary 62F03, 62M02, 26B10.

**Keywords:** parametric estimation; Taylor expansion; hypothesis testing; delta method; maximum likelihood estimation; moments method; Implicit Function Theorem.

# 1 Introduction

The traditional approach of dealing with parametric estimation consists in defining an estimator as the root or minimizer of a function and then to study its asymptotic

<footnote>*Université de Lorraine, CNRS, IECL, Inria, F-54000 Nancy, France; ORCID: 0000-0003-0406-9550; Antoine.Lejay@univ-lorraine.fr</footnote>

<footnote>†Université de Lorraine, CNRS, IECL, Inria, F-54000 Nancy, France; ORCID: 0000-0001-6187-2716; Sara.Mazzonetto@univ-lorraine.fr</footnote>
properties as the size of the observations increases. This is the case for the Maximum Likelihood Estimation (MLE, [28]) and its variants (pseudo- or quasi- likelihood estimation), or the Generalized Method of Moments which are used in a broad variety of problems [20, 27], or more generally for methods consisting in finding the zeros of a random function. A typical statement is that the estimator is consistent and asymptotically (mixed) normal: The properly renormalized error between the estimator and the unknown value converges in distribution to a (mixed) Gaussian random variable.

The limiting distribution may be used to quantify the quality of the estimation, and to construct for example confidence intervals. As pointed by D.A. Sprott [31, 32], the quality of the approximation depends more on the distance between the law of the estimator and the limiting distribution than on the size itself. Various tools have been introduced to improve the estimator for small or average sample size or to quantify its departure from its limiting distribution. Among them, let us cite Edgeworth expansions [29], Berry-Esséen inequality [8, 26, 35], Stein method [2, 3], jacknife [12], bootstrap [14], “magic formula” [5], Bartlett corrections [6, 10], and so on. Some of these corrections use considerations on the density of the estimator, others are pointwise ones.

Our main result, Theorem 3 in Section 3.3, is a recipe to get a pathwise expansion of the univariate root $\theta(n)$ of $F(n, \theta(n)) = 0$ — think of the MLE — of the form

$$ \theta(n) = \theta + H(n, G_n) $$

where $\theta$ is the true, unknown parameter, $G_n$ is asymptotical pivotal converging to some $G$ (think of the standard Gaussian random variable coming from a Central Limit Theorem) and $x \mapsto H(n, x)$ is a non-linear, random function of the kind

$$ H(n, x) = \sum_{k=1}^{p} \alpha_k(n) \varphi(n)^k x^k + O(\varphi(n)^{p+1}). \quad (1) $$

In the latter expansion, $\varphi(n)$ decreases to 0 as $n \to \infty$ and

$$ \{\alpha_k(n)\}_{k=1}^p \xrightarrow[n \to \infty]{\text{law or probability}} \{\alpha_k\}_{k=1}^p \quad (2) $$

for some $\alpha_k$'s. The $\alpha_k$'s and the $\alpha_k(n)$'s may depend on $\theta$. The expansions may have different forms for some particular values of $\theta$ for which some of the $\alpha_k$ vanishes (which we call a *phase transition*). We highlight that our approach is pathwise, and the order $p$ is related to the regularity of $x \mapsto F(n, x)$.

The family of coefficients $\{\alpha_k(n)\}_{k=1}^p$ in (1) is not unique. However, given a function $F(n, \cdot)$ and knowing the asymptotic behavior of $F(m, \cdot)$ and its high
order derivatives $D^k F(n, \cdot)$, we provide $\alpha_k(n)$'s satisfying (2) from $F(n, \cdot)$ and its derivatives (see Notation 3 for the formula and details in the Supplementary material).

Note that the expansion of $\theta(n)$ is such that

$$ \varphi(n)^{-1}(\theta(n) - \theta) \xrightarrow[n \to \infty]{\text{dist.}} c_1 G. $$

To obtain this limit, one simply needs to linearize $F$, thereby reducing the analysis to $F(\theta)$ and $F'(\theta)$. Many statisticians have focused on the normality of MLE [4] and on MLE as “method of generating approximate linear efficient pivots” [33]. The techniques often rely on changes of variables which make approximately constant the Fisher information [15] or cancel or minimize the second order derivative $F^{(2)}$ (skewness). As in these works, we discuss the role of (asymptotically) pivotal quantities. These quantities play a central role in the potentially nonlinear expansion of $\theta(n)$. The non-linearity can impact the convergence behavior of $\theta(n)$.

The expansion we provide may remind the reader of the Edgeworth expansions in [29] for the law of the MLE, but our result is pathwise.

Our approach is based on an asymptotic implicit function theorem (see Theorem 1 below), inspired from perturbative approaches. Performing asymptotic expansions has a long history with a broad range of applications, notably to study perturbations of (partial) differential equations and dynamical systems [11, 22, 24] or in change of scale analysis [7]. Although in one case, we recover the Lagrange inversion formula for power series [16, 17, 30], our starting point is different and seems not to have been considered in a general form but mostly for particular cases. Moreover, we also consider functions that are not necessarily analytic. Our approach seems to be new because it assumes weak general assumptions on the regularity of $F(n, \cdot)$ and takes into account different possible asymptotic behaviors of $D^k F(n, \cdot)$ (for statistical applications, think that $D^k F(n, \cdot)$ may converge thanks to the Law of Large Numbers (LLN) or to the Central Limit Theorem (CLT), hence with different rates). Actually, our result does not require anything more than knowing the asymptotic behavior of $F(n, \cdot)$ and its derivatives.

Theorem 3 is therefore a sort of generalization of the Delta method [34] in the sense that knowing the asymptotic behavior of the score and its derivative, we get the asymptotic behavior of a function of it, in particular the estimator. Besides, we do not restrict ourselves to a first order linearization.

We illustrate the ideas of (the proof of) our results through two examples in Section 2: first the Bernoulli model, for which we have no pretensions to say something new nor better than existing results, and then a far less trivial application, provided in [25], where we apply our procedure to the Skew Brownian motion (Skew-BM)
obtaining new expansions of the MLE estimator of the skewness parameter and
showing that it is asymptotically mixed normal.

The application to Skew-BM demonstrates that our approach is not limited to
independent samples or asymptotic normality. This flexibility is particularly
valuable for applications in the statistical analysis of diffusion processes or time
series. Moreover, our framework is not restricted to the Maximum Likelihood
Estimator (MLE); it also accommodates the Generalized Method of Moments,
pseudo- or quasi-likelihood estimators, and other variants (see, e.g., [27] and the
example in the Supplement).

Our procedure can also be used to study the lack of normality of the estimator
under a fixed sample size $n$ when $G$ is Gaussian. The lack of normality may
come from the fact that $G_n$ has not reached the Gaussian regime and/or by the
non-linearity of $H(n, \cdot)$. This is illustrated numerically in the simple case of the
exponential distribution as well as in the intricate case of the Skew-BM [25]. A
theoretical analysis would require the complementarity results of type Berry-Esseen
or Stein.

To conclude, let us mention that the main results of this document (in particular
Theorem 1) can be easily extended also for multidimensional parameters thanks to
multidimensional Inverse Function Theorem (see, e.g., [1]). It should also hold true
in an infinite dimensional setting, which should be useful to deal with semiparametric
models although there are some fine technical points to deal with (see the discussion
in [18, 19, 34]). In the multidimensional setting, a Taylor expansion is used in [21]
to compute the first two moments of functions of estimators. This might be
combined easily with our approach to obtain higher moments. We expect to
compute cumulants [13] as well in a wide variety of cases, and to identify some
corrections to enforce the quality of the estimation [34].

The supplementary material extends our work by presenting the results in a more
general setting. It provides detailed derivations of the coefficients {$\alpha_k(n)$}$_{k\ge0}$,
explores their properties, and examines their sensitivity to changes of variables
in $F(n, \cdot)$ and to perturbations, such as errors in the observations.

## 2 Examples and ideas

In this section, we illustrate our results that we provide in more generality in
Section 3. We first consider the simple example of distributions belonging to the
exponential family and in particular of exponential and Bernoulli distribution. In
the exponential family setting, the MLE is known explicitly and our expansion
does not yield any novelty. However, with the example of the Bernoulli model, we
show how one can derive our polynomial expansion for the MLE and illustrate its features. We show that the expansion relies on the knowledge of the score and on the asymptotic behaviors of its derivatives in form of a LLN or CLT. The notation of the main results of this paper is perfectly suited to capture this. Moreover, we relate our expansion to the Lagrange inversion formula. Furthermore, we discuss the impact of the possibly non-linear expansion on the asymptotic behavior of proxies of the estimator. Features as boundary layers effects and phase transitions are described as well.

Finally, we consider MLE for the parameter of Skew-BM, which is a stochastic process solution to a stochastic differential equation. The MLE is not explicitly known, the samples are not independent and identically distributed, the limit is not Gaussian, and the convergence rate is not $\sqrt{n}$. This context is one of the original motivation of this paper and the most important application so far.

## 2.1 An illustration through the exponential family

### 2.1.1 Setting and properties of the MLE

We illustrate our study by examining some properties of the MLE estimator for the exponential family with scalar parameter $\theta$ (see, e.g., [9, 28]). The density of the corresponding random variable $X$ with distribution $P_{\theta}$ is written as

$$p(x, \theta) = h(x) \exp(w(\theta)T(x) - A(\theta)) \quad (3)$$

for suitable functions $w$, $A$ and $T$. The function $h$ is used to normalize the density and does not depend on $\theta$. We assume that $w$ and $A$ are differentiable of class $C^{p+1}$ for some $p \ge 1$, and that $T(X)$ is square integrable under $P_{\theta}$.

For any two differentiable functions $S$ and $f$, we write $D_S f(x) := f'(x)/S'(x)$. When $S$ is the identity function, $S' \equiv 1$, we also write $D_x f(x) := f'(x)$.

The Fisher information is

$$\mathcal{J}(\theta) := w'(\theta)^2 D_w^2 A(\theta) = A^{(2)}(\theta) - w^{(2)}(\theta) D_w A(\theta). \quad (4)$$

We assume that $\mathcal{J}(\theta) \neq 0$.

The exponential models present particular features. Among them, $T(X)$ is a sufficient statistics that satisfies

$$\mathbb{E}_{\theta}[T(X)] = D_w A(\theta) \text{ and } \mathrm{Var}_{P_{\theta}}(T(X)) = D_w^2 A(\theta) = \mathcal{J}(\theta)/w'(\theta)^2.$$
For $(X_1, \dots, X_n)$ independent copies of $X$, we define

$$ T_n := \frac{1}{n} \sum_{i=1}^{n} T(X_i) \quad \text{and} \quad G_n := \sqrt{n} \frac{T_n - D_w A(\theta)}{\sqrt{D_w^2 A(\theta)}} = \sqrt{n} \frac{w'(\theta)T_n - A'(\theta)}{\sqrt{\mathcal{F}(\theta)}}. \quad (5) $$

With the LLN and the CLT,

$$ T_n \xrightarrow{n \to \infty} D_w A(\theta) \quad \text{and} \quad G_n \xrightarrow{n \to \infty} G \sim \mathcal{N}(0, 1) \quad \text{under } \mathbb{P}_{\theta}. $$

The likelihood of $(X_1, \dots, X_n)$ is the product of $p(X_i, \theta)$ as a function of $\theta$. The score, that is the derivative in $\theta$ of the log-likelihood, is

$$ F(n, \theta) := n(w'(\theta)T_n - A'(\theta)). $$

The MLE $\theta(n)$ solves $F(n, \theta(n)) = 0$.

Assuming that $D_w A$ is invertible, we obtain that $\theta(n) = (D_w A)^{-1}(T_n)$.

Some distributions in the exponential family are such that $D_w A(\theta) = \theta$ for all $\theta$ in the parameter space. In this case, $\theta(n) = T_n$ so that the distribution of $\theta(n)$ is explicitly known and its asymptotic behavior follows from the one of $T_n$. As $\sqrt{n}(\theta(n) - \theta) = G_n \sqrt{D_w^2 A(\theta)}$, the distance (Wasserstein, Kolmogorov-Smirnov,...) between $\sqrt{n}(\theta(n) - \theta)$ and a Gaussian distribution is equal to the distance between $G_n$ and $G$. Example of such distributions (with $T(x) = x$ unless explicitly specified) are

* Bernoulli distribution, in which case $n\theta(n) \sim \text{Bin}(n, \theta)$.

* Normal distribution with unknown mean $\theta$ and known variance $\sigma^2$, in which case $T_n - \theta \sim \mathcal{N}(0, \sigma^2/n)$.

* Laplace distribution with known mean $\mu$ and unknown scale $\theta$, in which case $\theta(n)/n\theta \sim \Gamma(n, 1)$, as $|X - \mu| \sim \mathcal{C}(\theta)$ and $T(x) = |x - \mu|$ [23].

* Poisson distribution with parameter $\theta$, in which case $n\theta(n) \sim \mathcal{P}(n\theta)$.

When $D_w A$ is not the identity, $\theta(n) = D_w A(T_n)$ expresses a possibly non-linear relationship between $T_n$, that converges in distribution, and the MLE $\theta(n)$. This may degrade the asymptotic behavior of the estimator with respect to the one of $T_n$. For example, when $X$ follows the exponential distribution with parameter $\theta$, $G_n = \sqrt{n}(\theta T_n - 1)$ and

$$ \theta(n) = \frac{1}{T_n} = \theta - \theta \frac{G_n}{\sqrt{n} + G_n} = \theta + \theta H\left(\frac{G_n}{\sqrt{n}}\right) \quad \text{with} \quad H(x) := \frac{-x}{1+x}, \quad (6) $$

while $n \cdot \theta \cdot T_n \sim \Gamma(n, 1)$. Since $G_n$ converges to $G$ in distribution, $\sqrt{n}(\theta(n) - \theta)$ converge to $\theta G$ in distribution. This is not only the case of the exponential distribution but also of Beta, Weibull, etc. The function $H$ has an impact on the rate of convergence of the MLE, as illustrated in Section 3.4.5.
### 2.1.2 Illustrating the results: Case of the Bernoulli distribution

We now present our idea on the Bernoulli distribution. As seen in the previous section, the MLE is just the empirical mean in this case, so what we are going to present is useless. Yet it allows one to show the main features of our method. Besides, it presents some close connections with the case of the Skew-BM shown below.

For the Bernoulli distributions with parameter $\theta \in (0, 1)$, in the notation of the previous section, we have

$$T(x) = x, w(\theta) = \log \frac{\theta}{1-\theta}, A(\theta) = -\log(1-\theta) \text{ and } \mathcal{F}(\theta) = \frac{1}{\theta(1-\theta)}.$$

We recall that $\mathcal{F}$ is the Fisher information. With $X_i \sim \text{Ber}(\theta)$ independent, $T_n := \sum_{i=1}^n X_i$ and $T_n = \theta + n^{-1/2}G_n\sqrt{\theta(1-\theta)}$.

**Asymptotic behavior of the score and its derivatives.** We define

$$\{F_{\theta}\}_0(n) := \frac{F(n, \theta)}{\sqrt{n}} = \sqrt{\mathcal{F}(\theta)}G_n,$$

$$\text{and } \{F_{\theta}\}_k(n) := \frac{D^k F(n, \theta)}{n} = w^{(k+1)}(\theta)T_n - A^{(k+1)}(\theta) \text{ for } k = 1, \dots, p.$$

Using the expressions of $w$ and $A$,

$$\{F_{\theta}\}_0(n) = \frac{G_n}{\sqrt{\theta(1-\theta)}} \text{ and } \{F_{\theta}\}_k(n) = k! \left( (-1)^k \frac{T_n}{\theta^{k+1}} - \frac{1-T_n}{(1-\theta)^{k+1}} \right).$$

Therefore, $\{F_{\theta}\}_0(n)$ converges in distribution to $\mu_0 := G/\sqrt{\theta(1-\theta)}$ with $G \sim \mathcal{N}(0, 1)$, while

$$\{F_{\theta}\}_k(n) \xrightarrow{n \to \infty} \mu_k := k! \left( \frac{(-1)^{k+1}}{\theta^k} + \frac{1}{(1-\theta)^k} \right).$$

**Taylor expansion of the score.** We set $\epsilon(n) := \theta(n) - \theta$. Using the Taylor expansion,

$$F(n, \theta(n)) - F(n, \theta) = \sum_{k=1}^{+\infty} \frac{D^k F(n, \theta)}{k!} \epsilon(n)^k. \qquad (7)$$

As $F(n, \theta(n)) = 0$ and $F(n, \theta)/\sqrt{n} = \{F_{\theta}\}_0(n)$, we then obtain by dividing (7) by $n$,

$$-\frac{\delta_0(n)}{\sqrt{n}} = \sum_{k=1}^{+\infty} \delta_k(n) \epsilon(n)^k \text{ with } \delta_k(n) := -\frac{1}{k!} \frac{\{F_{\theta}\}_k(n)}{\{F_{\theta}\}_1(n)}.$$
**MLE expansion.** Using a Lagrange inversion formula on the entire series $f_n(z) = \sum_{k \ge 1} \delta_k(n)z^k$, we obtain another entire series $\tilde{g}_n(z) = \sum_{k \ge 1} \tilde{\alpha}_k(n)z^k$ such that $f_n(\tilde{g}_n(z)) = -z$. This is possible because $f'_n(0) = \delta_1(n) = -1 \ne 0$. Therefore,

$$ \epsilon(n) = \theta(n) - \theta = \tilde{g}_n(\delta_0(n)/\sqrt{n}) = \sum_{k \ge 1} \tilde{\alpha}_k(n) \left( \frac{\delta_0(n)}{\sqrt{n}} \right)^k . $$

The term $\tilde{\alpha}_1(n)$ is equal to 1 as $\delta_1(n) = -1$. Besides,

$$ \delta_0(n) = G_n \frac{(\theta(1-\theta))^{3/2}}{\theta^2 + T_n(1-2\theta)} \xrightarrow{n \to \infty} G \sqrt{\theta(1-\theta)} $$

and

$$ \tilde{\alpha}_2(n) = \delta_2(n) = \frac{2 T_n((1-\theta)^3 + \theta^3) - \theta^3}{\theta(1-\theta) \theta^2 + T_n(1-2\theta)}. $$

Each $\tilde{\alpha}_k(n)$ is actually a polynomial in $\delta_\ell(n)$, $\ell = 1, \dots, k$. We then obtain an expression for the MLE as an entire function applied to $G_n/\sqrt{n}$. The coefficients, which are random, converge either is distribution or almost surely. We note that each coefficient may be expressed in term of $G_n/\sqrt{n}$. In particular,

$$ \theta(n) = \theta + \delta_0(n) + \sum_{k \ge 2} \tilde{\alpha}_k(n) \left( \frac{\delta_0(n)}{\sqrt{n}} \right)^k = \theta + G_n \sqrt{\theta(1-\theta)} + O\left(\frac{1}{n}\right). $$

Note that if we consider another entire series $g_n(z) = \sum_{k \ge 1} \alpha_k(n)z^k$ such that $f_n(g_n(z)) = -\delta_0(n)z$, then $\alpha_k(n) = \tilde{\alpha}_k(n)(\delta_0(n))^k$ and $\epsilon(n) = g(1/\sqrt{n})$.

**Property: boundary layer effect.** One of the potential difficulties lies in the presence of a *boundary layer*, meaning that for $\theta$ close to 0 or 1, the terms $\alpha_k(n)$ growth quickly. Each $\alpha_k(n)$ is a polynomial in $\delta_1(n), \dots, \delta_k(n)$ and contains as term $\delta_2(n)^k$. Yet

$$ \delta_2(n) = \frac{2(2\theta^2 - 3\theta + 1)}{\theta(1-\theta)^2} + O(G_n/\sqrt{n}). $$

Therefore, if $\theta(1-\theta)$ has a magnitude comparable to $G_n/\sqrt{n}$, the series giving $\theta(n)$ has a low rate of convergence.

**Property: phase transition.** We now consider that $\theta = 1/2$ and we show that a sort of *phase transition* appears: another expansion of the MLE.

For $\theta = 1/2$, then $\mu_{2k} = 0$. Hence, changing the definition of $\{F_\theta\}_{2k}(n)$ for even indices,

$$ \{F_{\theta=1/2}\}_{2k}(n) := \frac{D^{2k} F(n, 1/2)}{\sqrt{n}}, $$
we obtain thanks to the CLT that

$$ \{F_{\theta=1/2}\}_{2k}(n) = \sqrt{n}\frac{(2k)!}{2^{2k+1}}(2T_n - 1) \xrightarrow[n \to \infty]{\text{dist.}} \frac{(2k)!}{2^{2k+1}}\sqrt{NG}. $$

Considering again the Taylor expansion (7), we obtain again that

$$ -\frac{\{F_{\theta=1/2}\}_0(n)}{\sqrt{n}} = \frac{1}{\sqrt{n}} \sum_{k=1}^{+\infty} \frac{D^{2k} F(n, 1/2)}{\sqrt{n}(2k)!} \epsilon(n)^{2k} + \sum_{k=0}^{+\infty} \frac{D^{(2k+1)} F(n, 1/2)}{n(2k+1)!} \epsilon(n)^{2k+1}. $$

Introducing the coefficients $\{F_{\theta=1/2}\}_k$, and dividing the expression by $\{F_{\theta=1/2}\}_1 \neq 0$,

$$ -\frac{\delta_0(n)}{\sqrt{n}} = \frac{1}{\sqrt{n}} \sum_{k=1}^{+\infty} \delta_{2k}(n) \epsilon(n)^{2k} + \sum_{k=0}^{+\infty} \delta_{2k+1}(n) \epsilon(n)^{2k+1}. $$

We now write

$$ f_{\text{even}}(x) = \sum_{k=1}^{+\infty} \delta_{2k}(n)x^{2k} \quad \text{and} \quad f_{\text{odd}}(x) = \sum_{k=0}^{+\infty} \delta_{2k+1}(n)x^{2k+1} $$

and we are looking for a function $g(x) = \sum_{k=1}^{+\infty} \alpha_k(n)x^k$ such that

$$ -x\delta_0(n) = f_{\text{even}}(g(x)) + x f_{\text{odd}}(g(x)). $$

The problem of finding $g$ leads to a different formula from the one given by the Lagrange inversion formula. Yet it is still tractable so that each $\alpha_k(n)$ can be computed in a recursive manner from $\delta_0(n), \dots, \delta_k(n)$. We then set

$$ \epsilon(n) = g\left(\frac{1}{\sqrt{n}}\right). $$

**Property: suitable change of scale.** Let us note, in the setting of the exponential family, that considering $\tilde{F}(n, \theta) = F(n, \theta)/w'(\theta)$ leads to $\delta_k(n)$ which are not random. In the Bernoulli model, $\tilde{F}(n, \theta) = n(T_n - \theta)$, so our procedure leads to $\theta(n) = \theta + \frac{\sqrt{\theta(1-\theta)}}{\sqrt{n}}G_n = T_n$. This suggests that considering change of variables in the likelihood or in the score, may lead to simpler expressions for the estimator.

## 2.2 Skew Brownian motion

A far less trivial application is provided in [25] where we apply our procedure to the Skew-BM obtaining new expansions of the MLE estimator of the skewness parameter and showing that the MLE is asymptotically mixed normal.
We hereby show that our approach is not restricted to iid samples, not to convergence whose rate is 1/2.

The Skew-BM $X$ of parameter $\theta \in [-1, 1]$ solves the stochastic differential equation

$$X_t = B_t + \theta L_t, \quad t \ge 0 \qquad (8)$$

for $B$ a Brownian motion and $\{L_t\}_{t \ge 0}$ its symmetric local time of $X$ at point 0. The Skew-BM $\{X_t\}_{t \in [0, T]}$ of parameter $\theta_0 \in (-1, 1)$ is observed at times $\{t_i\}_{i=0, \dots, \lfloor nT \rfloor}$ on a time window $[0, T]$ with $t_i := i/n$. Hence $n$ is the number of samples per unit of time.

The score is $F(n, \theta) = \sum_{i=0}^{\lfloor nT \rfloor - 1} k_{\theta}(X_{t_i} \sqrt{n}, X_{t_{i+1}} \sqrt{n})$, where

$$k_{\theta}(x, y) = \frac{\operatorname{sgn}(y)}{\operatorname{sgn}(y)\theta + \exp(2\max\{xy, 0\})}.$$

The asymptotic behavior of the MLE estimator of the parameter $\theta$ is related to the $m$-th derivatives of the score

$$\{F\}_0(n) = \frac{1}{n^{1/4}} F(n, \theta),$$

$$\{F\}_m(n) = \frac{1}{\sqrt{n}} D^m F(n, \theta) = \frac{1}{n^{1/2}} \sum_{i=0}^{\lfloor nT \rfloor - 1} \partial_{\theta}^m k_{\theta}(X_{t_i} \sqrt{n}, X_{t_{i+1}} \sqrt{n}).$$

The coefficient $\{F\}_0(n)$ converges stably in law to $\xi_0(\theta) \sqrt{L_T H}$ where $H$ is a Gaussian independent of the Skew-BM and of $L_T$, which is its local time. For each $m \ge 1$, $\{F\}_m(n)$ converges in probability to $\xi_m(\theta) L_T$. The convergence is actually joint.

Note that if $\theta = 0$, then the following quantity has a non trivial limit

$$\{F\}_{2m}(n) = \frac{1}{n^{1/4}} D^m F(n, \theta) = \frac{1}{n^{1/4}} \sum_{i=0}^{\lfloor nT \rfloor - 1} \partial_{\theta}^m k_{\theta}(X_{t_i} \sqrt{n}, X_{t_{i+1}} \sqrt{n}).$$

The asymptotic behavior, as $n \to \infty$, of these quantities depends on a Gaussian vector independent of the process.

As in Section 2.1.2, the Taylor expansion (7) and the Lagrange inversion formula yield an expansion of $\theta(n)$ based on the score and its derivatives which allow us to prove that $n^{1/4}(\theta(n) - \theta_0)$ converges towards a mixed Gaussian law. We present the general procedure in this article, which, in the case of Skew-BM, it allowed to understand the semi-asymptotic behavior of the MLE for the skewness parameter, and to prove its asymptotic behavior for the first time [25]. Boundary layers ($|\theta_0|$ close to 1) and phase transition (when $\theta_0 = 0$) are present in this case as well.
# 3 Main results

In this section we consider the inference of a parameter $\theta_0 \in \mathbb{R}$ contained in the space of parameters $\Theta \subseteq \mathbb{R}$.

Our purpose is to provide polynomial approximations $\theta_p(n) \in \Theta$ of the unknown estimator $\theta(n) \in \Theta$ of $\theta_0$ satisfying $F(n, \theta(n)) = 0$. If $F(n, \theta_p(n))$ is small, we deduce the distance of $\theta_p(n)$ from $\theta(n)$. This is firstly a pathwise result, analytical and not stochastic. Randomness comes into play when considering the asymptotic properties leading to our main result: Theorem 3 in Section 3.3. However, to introduce some notations, we may refer to the desired probabilistic properties already in the following preliminary sections.

## 3.1 Notations and assumptions

**Hypothesis 1.** The space of parameters $\Theta$ contains an open interval around $\theta_0$.

**Notation 1.** For any $p \in \{1, \dots, +\infty\}$ and any sequence of functions $F(n, \cdot) \in \mathcal{C}^p(\Theta, \mathbb{R})$, $n \in \mathbb{N}$, we denote by $D^m$ the derivative of order $m$ with respect to $\theta$ of $F(n, \theta)$.

The idea of the following assumptions and notations is that the $\beta_m$ are such that $\varphi(n)^{\beta_m} D^m F(n, \theta_0)$ converges as $n$ goes to infinity. In the examples of the previous section $\varphi(n) = n^{-1/2}$ (exponential family) or $\varphi(n) = n^{-1/4}$ (Skew-BM) and $\beta_m$ was either equal to 1 or to 2 as in the following assumptions.

**Hypothesis 2.** Let $\varphi : \mathbb{N} \to (0, 1)$ be a function, called *rate function*. Consider a sequence $\beta = \{\beta_k\}_{k \ge 0}$ to be one of the following sequences:

$$ \beta^{-} := (1, 2, 2, 2, \dots, 2, 2, \dots) \text{ and } \beta^{\wedge} := (1, 2, 1, 2, \dots, 1, 2, \dots). $$

We refer to these choices of $\beta$ respectively as “up-flat” and “zigzag” sequence.

Referring to the previous section, both $\beta^{-}$ and $\beta^{\wedge}$ have been considered for Bernoulli model and Skew-BM. Indeed, $\beta^{\wedge}$ intervenes when the Bernoulli parameter is $\theta_0 = 1/2$ and the skewness parameter $\theta_0 = 0$.

**Notation 2.** For any $p \in \{1, \dots, +\infty\}$ and any sequence of functions $F(n, \cdot) \in \mathcal{C}^p(\Theta, \mathbb{R})$, $n \in \mathbb{N}$, let for all $k \in \{0, \dots, p\}$,

$$ \{F\}_k(n) := \varphi(n)^{\beta_k} D^k F(n, \theta_0). $$

For every $n \in \mathbb{N}$ such that $DF(n, \theta_0) \neq 0$, let $\delta^F(n)$, the vector of $\mathbb{R}^{p+1}$ defined by

$$ \delta^F(n) := \left\{ -\frac{1}{k!} \{F\}_k(n) \right\}_{k=0,\dots,p}. $$
As mentioned above, the idea behind the notation is that {$F$}$_m(n) = \varphi(n)^{\beta_m} D^m F(n, \theta_0)$ converges as $n$ goes to infinity, and so does $\delta^F(n)$. We are now ready to introduce $\alpha = \{\alpha_k\}_{k \ge 1}$, related to $\delta^F(n)$, that satisfies $\theta(n) - \theta_0 = \sum_{k \ge 1} \alpha_k \varphi(n)^k$.

**Notation 3** ($\alpha^{-}(\delta), \alpha^{\sim}(\delta)$). Given $p \in \{1, \dots, +\infty\}$ and a sequence $\delta = \{\delta_k\}_{k=0,\dots,p}$ such that $\delta_1 = -1$, let $\alpha^{-}(\delta) = \{\alpha_k\}_{k=0,\dots,p}$, be such that

$$ \alpha_0 = 0, \quad \alpha_1 = \delta_0 \quad \text{and} \quad \alpha_q = \sum_{m=2}^{q} \delta_m \sum_{k_1+\dots+k_m=q} \alpha_{k_1} \cdots \alpha_{k_m} \text{ for } 2 \le q \le p, \quad (9) $$

and, for $\alpha^{\sim}(\delta) = \{\alpha_k\}_{k=0,\dots,p}$, take $\alpha_1 = \delta_0$, $\alpha_{2q} = 0$ for $2q \le p$ and for $2q + 1 \le p$

$$ \alpha_{2q+1} = \sum_{\substack{m=2 \\ m \text{ even}}}^{2q} \delta_m \sum_{k_1+\dots+k_m=2q} \alpha_{k_1} \cdots \alpha_{k_m} + \sum_{\substack{m=3 \\ m \text{ odd}}}^{2q+1} \delta_m \sum_{k_1+\dots+k_m=2q+1} \alpha_{k_1} \cdots \alpha_{k_m}. \quad (10) $$

*Remark 1.* Let us give more explicit expressions, for $p = 5$.

For $\alpha^{-}(\delta)$, we have $\alpha_0 = 0$, $\alpha_1 = \delta_0$ and

$$ \begin{aligned} \alpha_2 &= \delta_2 \delta_0^2, & \alpha_3 &= (\delta_3 + 2\delta_2^2)\delta_0^3, \\ \alpha_4 &= (\delta_4 + 5\delta_2\delta_3 + 5\delta_2^3)\delta_0^4, & \alpha_5 &= (\delta_5 + 6\delta_2\delta_4 + 21\delta_2^2\delta_3 + 14\delta_2^4 + 3\delta_3^2)\delta_0^5. \end{aligned} $$

For $\alpha^{\sim}(\delta)$, we have $\alpha_0 = 0$, $\alpha_1 = \delta_0$ and

$$ \begin{aligned} \alpha_2 &= 0, & \alpha_3 &= \delta_3 \delta_0^3 + \delta_2 \delta_0^2, \\ \alpha_4 &= 0, & \alpha_5 &= (\delta_5 + 3\delta_3^2)\delta_0^5 + (5\delta_2\delta_3 + \delta_4)\delta_0^4 + 2\delta_2^2\delta_0^3. \end{aligned} $$

Note that $\alpha_{2k+1}$ is not homogeneous in $\delta_0^{2k+1}$ but contains terms in $\delta_0^j$ for $j \in \{k+1, \dots, 2k+1\}$.

*Remark 2.* Let $p \in \{1, \dots, +\infty\}$, a sequence $\delta = \{\delta_k\}_{k=0,\dots,p}$, $\bullet \in \{-, \sim\}$, and $\beta^\bullet = \{\beta_k\}_{k=0,\dots,p}$. Then the sequence $\alpha^\bullet(\delta) = \{\alpha_k\}_{k=0,\dots,p}$ satisfies for all $z \ne 0$ that

$$ \delta_0 z^{-1} + \sum_{m=1}^{p} \delta_m \sum_{k_1+\dots+k_m<p-1+\beta_m} \alpha_{k_1} \cdots \alpha_{k_m} z^{\sum_{n=1}^{m} k_n - \beta_m} = 0. \quad (11) $$

Note that, under the assumption {$F$}$_1(n) \ne 0$, (11) holds with $\alpha^\bullet(\delta^F(n))$ and both $\delta = \delta^F(n)$ and $\delta = \{{\{F\}}_k(n)/k!\}_{k=0,\dots,p}$.

The $\alpha$ have been computed via series inversion method, as in Section 2. The supplementary material provides the proof in a more general setting, allowing for different choices of $\beta$.

**Notation 4.** In a systematic manner, we use sums over multi-indices satisfying some constraints. Such multi-indices are denoted by $(k_1, \dots, k_m)$ and each $k_i$ is an integer.
## 3.2 The asymptotic implicit function theorem

We now state a series expansion which approximates the roots of $F(n, \cdot)$. We have not introduced randomness yet.

**Theorem 1 (Approximation of roots).** Let $p \in \{1, 2, \dots\}$ and assume that Hypotheses 1 and 2 hold. In particular, $\beta = \beta^*$ with $\bullet \in \{\nearrow, \nwarrow\}$. Let $n \in \mathbb{N}$ such that $F(n) \in \mathcal{C}^{p+1}(\Theta, \mathbb{R})$ and there exists a convex neighborhood $U(n)$ of $\theta_0$ in $\Theta$ satisfying:

(i) There exists $\theta(n) \in U(n)$ such that $F(n, \theta(n)) = 0$.

(ii) It holds that

$$|\varphi(n)|^2|DF(n, w)| \ge c(n) > 0, \text{ for all } w \in U(n) \\ \text{and } \max_{m \in \{2, \dots, p+1\} \ w \in U(n)} \sup_{w \in U(n)} |D^m F(n, w)| < +\infty.$$

(iii) It holds that $\theta_0 + \sum_{i=1}^p |\alpha_i(n)| \cdot |\varphi(n)|^i \in U(n)$, where $\{\alpha_k(n)\}_{k=0, \dots, p} = \alpha^*(\delta^F(n))$. Then

$$|\theta(n) - \theta_0 - \sum_{i=1}^{p} \alpha_i(n) \varphi(n)^i| \le K^*(n) \max \left\{ \sum_{k=1}^{p} |\alpha_k(n)|, \left( \sum_{k=1}^{p} |\alpha_k(n)| \right)^{p+1} \right\} |\varphi(n)|^{p+1},$$

with

$$K^-(n) \le \frac{1}{c(n)} \max_{m \in \{2, \dots, p+1\} \ w \in U(n)} \sup_{w \in U(n)} |\varphi(n)|^2 |D^m F(n, w)| < \infty,$$

if $\beta = \beta^-$ or $p=1$ and $\beta = \beta^{\sim}$, otherwise (if $p \ge 2$ and $\beta = \beta^{\sim}$)

$$K^{\sim}(n) \le K^-(n) + \frac{1}{c(n)} \max_{\substack{m \in \{1, \dots, p\} \\ \text{m even}}} |\{F\}_m(n)| < \infty.$$

The proof of Theorem 1 is provided in Appendix A. We also consider the case of analytic functions, in which the expansion is infinite.

**Theorem 2 (Approximation of roots, analytic case).** Let $p = +\infty$ and assume that Hypotheses 1 and 2 hold. In particular, $\beta = \beta^*$ with $\bullet \in \{\nearrow, \nwarrow\}$. Let $F: \mathbb{N} \times \Theta \to \mathbb{R}$ be such that $F(n, \cdot)$ is analytic. Let $n \in \mathbb{N}$ such that there exist a convex neighborhood $U(n)$ of $\theta_0$ in $\Theta$ satisfying:

(i) There exists $\theta(n) \in U(n)$ such that $F(n, \theta(n)) = 0$.

(ii) It holds that $|\varphi(n)|^2|DF(n, w)| \ge c(n) > 0$ for all $w \in U(n)$.

(iii) It holds that $\theta_0 + \sum_{i=1}^{+\infty} |\alpha_i(n)| \cdot |\varphi(n)|^i \in U(n)$ where $\{\alpha_k(n)\}_{k=0, \dots, p} = \alpha^*(\delta^F(n))$. Then

$$\theta(n) = \theta_0 + \sum_{i=1}^{+\infty} \alpha_i(n) \varphi(n)^i.$$
### 3.3 The stochastic asymptotic implicit function theorem

**Hypothesis 3.** Let $(F(n, \theta))_{(n,\theta)\in\mathbb{N}\times\Theta}$ be a family of random functions on a probability space $(\Omega, \mathcal{F}, \mathbb{P})$, let $p \in \{1, \dots, +\infty\}$, and let $\varphi$ and $\beta = \beta^\bullet$ with $\bullet \in \{\nearrow, \nwarrow\}$ as in Hypothesis 2.

**Proposition 1.** Assume Hypothesis 3 holds. Assume that for all $n \in \mathbb{N}$ and $\mathbb{P}$-almost all $\omega \in \Omega$ the function $\theta \mapsto F(n, \theta)$ is of class $\mathcal{C}^p$ at $\theta_0$, and there exists a random vector $\mu = \{\mu_k\}_{k=0}^p$ such that

$$ (\{F\}_0(n), \dots, \{F\}_p(n)) \xrightarrow{n \to \infty} \mu. $$

Then

$$ (i) \quad \delta^F(n) \xrightarrow{n \to \infty} \delta := \left\{-\frac{1}{m!} \frac{\mu_m}{\mu_1}\right\}_{m=0,\dots,p}. $$

$$ (ii) \quad \alpha^\bullet(\delta^F(n)) \xrightarrow{n \to \infty} \alpha^\bullet(\delta). $$

**Definition 1 (Boundedness in probability).** A sequence random variables $\{X_n\}_{n\in\mathbb{N}}$ is said to be *bounded in probability* if for any $\epsilon > 0$, there exist $K \ge 0$ and $n \in \mathbb{N}$ such that $\sup_{n \le m} \mathbb{P}[|X_m| \ge K] \le \epsilon$.

**Hypothesis 4.** Assume Hypothesis 3. Assume that there exists an open neighborhood $U \subseteq \Theta$ of $\theta_0$ such that for all $n \in \mathbb{N}$ and $\mathbb{P}$-almost all $\omega \in \Omega$:

(i) $\theta \mapsto F(n, \theta)$ is of class $\mathcal{C}^{p+1}$ in $U$.

(ii) There exists $\theta(n) \in U$ such that $F(n, \theta(n)) = 0$.

(iii) $DF(n, \theta) \ne 0$ for all $\theta \in U \setminus \{\theta_0\}$ and $\{1/\{F\}_1(m)\}_{m\in\mathbb{N}}$ is bounded in probability.

(iv) The sequence $\{N_m(F)\}_{m\in\mathbb{N}}$ is bounded in probability with

$$ N_m(F) := \sup_{1 \le k \le p+1} \sup_{\theta \in U} |\varphi(n)^2 D^k F(n, \theta)|. $$

The following proposition is a direct consequences of Theorem 1.

**Theorem 3.** Assume Hypothesis 4 and that the rate function $\varphi: \mathbb{N} \to (0, 1)$ is a monotone decreasing function such that $\lim_{n \to \infty} \varphi(n) = 0$. Assume that there exists a random vector $\mu = \{\mu_k\}_{k=0}^p$ such that

$$ (\{F\}_0(n), \dots, \{F\}_p(n)) \xrightarrow{n \to \infty} \mu, $$

and for every $n \in \mathbb{N}$ let $\alpha(n) := \alpha^\bullet(\delta^F(n))$. Let $\alpha := \alpha^\bullet(\delta)$ and $\delta$ as in Proposition 1. We introduce

$$ P_{\ell}(\varphi(n); \alpha(n)) := \sum_{k=0}^{\ell} \alpha_k(n) \varphi(n)^k, \quad (12) $$

$$ \theta_{\ell}(n) := \theta_0 + P_{\ell}(\varphi(n), \alpha(n)) \quad \text{and} \quad \theta_{\ell}(n, \infty) := \theta_0 + P_{\ell}(\varphi(n), \alpha) \quad (13) $$

for $\ell = 0, \dots, p$. Then,
(i) $\theta_p(n) - \theta_0 = P_p(\varphi(n); \alpha(n))$ converges to 0 in probability.

(ii) $\varphi(n)^{-(p+1)}|\theta(n) - \theta_p(n)| = \varphi(n)^{-(p+1)}|\theta(n) - \theta_0 - P_p(\varphi(n); \alpha(n))|$ is bounded in probability.

(iii) $\varphi(n)^{-(k+1)}(\theta_p(n) - \theta_k(n)) = \varphi(n)^{-(k+1)}(P_p(\varphi(n); \alpha(n)) - P_k(\varphi(n); \alpha(n))) \xrightarrow{dist.} \alpha_{k+1}$ and

$$\varphi(n)^{-(k+1)}(\theta(n) - \theta_k(n)) \xrightarrow{s \to \infty} \alpha_{k+1}$$

for $k \in \{0, \dots, p-1\}$ (with $P_0 = 0$). In particular if $k = 0$

$$\varphi(n)^{-1}(\theta(n) - \theta) \xrightarrow{s \to \infty} \alpha_1.$$

(iv) $\theta(n) - \theta_p(n, \infty)$ converges to 0 in probability with rate $\varphi(n) (\alpha_1(n) - \alpha_1)$.

## 3.4 Comments and conclusions

Let us comment on our result while relating it to standard situations.

In the standard situation, such as the one of the exponential family, the LLN ensures that $\{F\}_0(n)$ converges to the parameter and the CLT ensures that $\{F\}_0(n)$ behaves like a Gaussian centered in the parameter whose variance is the limit (ensured by the LLN) of $\{F\}_1(n)$. Note that the limit of $\{F\}_1(n)$ is the Fisher information. Indeed, in our assumptions $\{F\}_1$ never vanishes. The quantities $\{F\}_0(n)$ and $\{F\}_1(n)$ lead to $\alpha_1(n) = \delta_0(n) = -\{F\}_0(n)/\{F\}_1(n)$.

The quantity $\alpha_1(n)$ is crucial in determining the asymptotic behavior of the estimator and in the estimator expansions proposed in Theorem 3. The latter theorem exhibits two approximations of $\theta(n)$, i.e.,

$$\theta_p(n) := \theta_0 + P_p(\varphi(n), \alpha^*(n)) \quad \text{and} \quad \theta_p(n, \infty) := \theta_0 + P_p(\varphi(n), \alpha^*). \quad (14)$$

### 3.4.1 Importance of $\alpha_1(n)$ and pivotal quantities

Recall that $\alpha_1(n) = \delta_0(n) = -\{F\}_0(n)/\{F\}_1(n)$. The estimator satisfies

$$\theta(n) = \theta_0 + \alpha_1(n)\varphi(n) + O(\varphi(n)^2),$$

which tells that $\alpha_1(n)$ determines the asymptotic behavior of $\theta(n) - \theta_0$.

In the examples in Section 2, for $n$ large enough, $\alpha_1(n)^2\{F\}_1(n)$ (alternatively $\alpha_1(n)^2\mu_1$ and $\{F\}_0(n)^2/\{F\}_1(n)$ and $\{F\}_0(n)^2/\mu_1$) appears to be asymptotically “pivotal”, in the sense that its distribution no longer depends on the parameter. We could say it is the Wald's statistics. Hence, for instance, we could change (15) to express the MLE as a non-linear transform of one of these pivotal quantities. A further direction of research is to use this to design confidence intervals or hypothesis tests that takes the lack of normality into account.
### 3.4.2 Importance of $\alpha_1(n)$ in $\theta_p(n)$

First note that $\theta(n) = \theta_p(n) + O(\varphi(n)^{p+1})$.

Observe that the upper error bound in Theorem 1 (non-random version of Theorem 3) also depends on $\alpha_1(n)$. Therefore, when $\alpha_1(n)$ deviates from 0, we do not expect the truncated expansions $\theta_p(n)$ to be accurate.

Moreover, the variables $\alpha_k(n)$, $k \ge 2$, are polynomial expressions in $\alpha_1(n)$ and $\delta_k^F(n) = -\{F\}_k(n)/(k!\{F\}_1(n))$ for $k \ge 1$. Hence, in the case “up-flat”, owing to the homogeneity in $\alpha_1(n)$, granted by (9),

$$ \theta_p(n) = \theta_0 + \sum_{k=1}^{p} \xi_k(n) \alpha_1(n)^k \varphi(n)^k, $$

where the coefficients $\xi_k(n)$, $k \ge 2$ are polynomials expressions in the terms $\delta_k^F(n)$, $1 \le l \le k$, and $\xi_1(n) = 1$. We shorten this as

$$ \theta_p(n) = \theta_0 + H_p(n, \alpha_1(n)\varphi(n)), \quad (15) $$

where $H_p(n, \cdot)$ is a non-linear (polynomial or analytic) random function with $H_p(n, x) = x + O(x^2)$.

In many practical situations, as in the case of models in Section 2, $\alpha_1(n)$ fluctuates asymptotically as a (mixed) Gaussian random variable, while the coefficients of the expansion $H_p(n, \cdot)$ converges in probability to constants.

A similar result holds for the case “zig-zag” ($\beta = \beta^{\sim}$), excepted that the expression is no longer polynomial in $\alpha_1(n)\varphi(n)$, but still polynomial in $\alpha_1(n)$ and $\varphi(n)$ (granted by (10)). Some of the coefficients of $H_p$ converges in probability to (mixed) Gaussian distributions.

### 3.4.3 Non-uniqueness of the expansion

We illustrated that the expansion is not unique via the MLE in the setting of the Bernoulli model. We have seen that for specific values of the parameter to estimate, sometimes several choices of $\beta$ are possible (“up-flat” and “zig-zag”), leading to different finite approximation of the estimator. We call this phenomena phase transition, and we illustrated this, not only for Bernoulli model, but also for the Skew-BM in Section 2.

Sometimes suitably changing $F(n, \cdot)$, via a change of variable or a transformation preserving the zeros, provides an exact expression of $\theta(n)$ instead of an approximation (e.g., taking $\tilde{F}(n, \theta)$ for the Bernoulli model).
### 3.4.4 Semi-asymptotic behavior

As above, in the case “upflat”, we have the following convenient rewriting:

$$ \theta_p(n) = \theta_0 + \sum_{k=1}^{p} \xi_k(n) \alpha_1(n)^k \varphi(n)^k, \quad \text{and} \quad \theta_p(n, \infty) = \theta_0 + \sum_{k=1}^{p} \xi_k \alpha_1^k \varphi(n)^k, $$

where $\alpha_1(n)$ and $\xi_k(n)$ (i.e. $H_p(n, \cdot)$ in (15)) have been replaced by their limits. The expansion $\theta_p(n, \infty)$ has the advantage that the coefficients $\xi_k$ are simpler than $\xi_k(n)$, in the sense that $\xi_k(n)$ are random polynomials in $\varphi(n)$.

We discuss now these “proxies” of the behavior of the MLE. The expansion $\theta_p(n, \infty)$ is useful when the convergence of $\alpha_k(n)$’s toward their limits is faster than the one of $\theta_p(n)$. An example in which this does not work is the Bernoulli model (not in natural scale). Instead we have two examples for which this works well: the Skew-BM in [25] and the exponential distribution which we detail in Section 3.4.5 below. The take-away message is that the higher derivatives of the score may impact the convergence, e.g., in the case of the exponential distribution where the intrinsic non-linearity of the estimator can be captured by higher order derivatives.

### 3.4.5 Numerical illustration of the asymptotic behavior

To write down some results of type CLT for the MLE, it is sufficient to consider the asymptotic behavior score and its first order derivative. However, constant in the convergence are impacted by the asymptotic behavior of the high-order derivatives of the score.

Two numerical examples illustrate this fact.

For the exponential distribution, we saw that, when the true parameter is $\theta$,

$$ \theta(n) = \theta - \theta \frac{G_n}{\sqrt{n} + G_n} \quad \text{with} \quad G_n = \sqrt{n}(\theta T_n - 1) = -\frac{\alpha_1(n)}{\theta}. $$

We remark that $G_n > -\sqrt{n}$, so that the MLE is well defined. We saw that $G_n$ is asymptotically pivotal, in the sense that $G_n \xrightarrow{n \to \infty} G \sim \mathcal{N}(0, 1)$, so that the limit of $G_n$ does not depend on $\theta$. We define

$$ \varepsilon(n) := \sqrt{n} \frac{\theta(n) - \theta}{\theta} \xrightarrow{n \to \infty} G. $$

Using the function $H(x) := -x/(1+x)$ already introduced in (6),

$$ \varepsilon(n) = \sqrt{n} H \left( \frac{G_n}{\sqrt{n}} \right). $$
By applying our expansion

$$ \theta_{\infty}(n) = \theta_{\infty}(n, \infty) = \theta \sum_{k=0}^{+\infty} \frac{(-G_n)^k}{n^{k/2}}, $$

we recover the series expansion of $H(x) = \sum_{k \ge 1} (-x)^k$ for $|x| < 1$. Within the exponential family, the high-order derivatives of order $k$ of the scores are affine functions of $T_n$. For the exponential distribution, they are actually deterministic functions for $k \ge 2$, so that $\theta(n) = \theta_{\infty}(n)$ (see (14)), and $\alpha_k'(n) = \alpha_k'(n)$ for $k \ge 2$.

As $G_n/\sqrt{n}$ converges in probability to 0, for any $K > 0$,

$$ \mathbb{P}\left[H\left(\frac{G_n}{\sqrt{n}}\right) + \frac{G_n}{\sqrt{n}} \ge K\right] = \mathbb{P}\left[\frac{(G_n/\sqrt{n})^2}{1 + G_n/\sqrt{n}} \ge K\right] \underset{n \to \infty}{\longrightarrow} 0. $$

Therefore, we could use $G_n$, $\sqrt{n}H(G/\sqrt{n})$ and $G$ as approximations of $\varepsilon(n)$ when $n$ is large, but fixed. To quantify this, let $\Delta$ be the Kolmogorov-Smirnov distance: For any two random variables $X_1, X_2$, let

$$ \Delta(X_1; X_2) := \sup_{x \in \mathbb{R}} |\mathbb{P}[X_1 \le x] - \mathbb{P}[X_2 \le x]|. $$

We then define

$$ \begin{aligned} \Delta_1 &:= \Delta(\varepsilon(n), G), \\ \Delta_2 &:= \Delta(\varepsilon(n), \sqrt{n}H(G/\sqrt{n})), \end{aligned} \quad (16) $$

$$ \text{and } \Delta_3 := \Delta(\sqrt{n}H(G/\sqrt{n}), G). $$

By the triangle inequality, $\Delta_1 \le \Delta_2 + \Delta_3$. When $\Delta_1$ is small, we could for example build a Wald confidence interval using the Gaussian distribution $G$. Since $H$ is invertible, $\Delta_2 = \Delta(G_n, G)$, up to neglecting $\{G \le -\sqrt{n}\}$. When $\Delta_2$ is small, one could replace $G_n$ by its Gaussian limit $G$.

These distances are computed using $10^6$ samples of $T_n$. The Kolmogorov-Smirnov statistics allows one to assert that the Monte Carlo error, of order $10^{-3}$, is small in front of the $\Delta_i$. We see from Figure 1 that $\Delta_2$ is around half $\Delta_3$ (which does not depend on $\theta$). This means that the convergence of $G_n$ toward $G$ is faster (same rate but smaller constant) than the convergence of $\theta(n)$ toward $\theta$.

When $n$ is small, it is far more satisfactory to use $\sqrt{n}H(G/\sqrt{n})$ than $G$ to construct for example confidence intervals or tests for the value $\theta$.

With the **Skew Brownian motion**, we use for an approximation $\theta_p(n, \infty)$, meaning that we replace $\alpha_k'$ by its limits: When $L_T$ is the local time at 0 and time $T$ of the Skew-BM, the coefficient $\alpha_1$ is a mixed Gaussian distribution of type $cG/\sqrt{L_T}$,
<figure><img src="image_3.png" /><figcaption>Figure 1: Kolmogorov-Smirnov distances for $\theta = 1$ for the exponential distribution.</figcaption></figure>

while the other coefficients are proportional to $L_T$ when $\theta \neq 0$. When $\theta = 0$, coefficient at odd indices are themselves mixed Gaussian.

As for the exponential distribution, we see that the convergence of the MLE is slower (same rate but slower constant) than the one of the score and its first derivative (linked to the Fisher information). Again, the semi-asymptotic behavior of the MLE cannot be deduced from the asymptotic behavior of the score and its derivative. Other numerical experiments, shown in Figure 2 and in [25, Figure 3], illustrates that the approximations $\theta_5(n)$ (true coefficients $\alpha'^{-}(n)$) and $\theta_5(n, \infty)$ (using the limits coefficients $\alpha'^{-}$) provide suitable approximations of the MLE, while $\theta_1(n)$ — the approximation of $\theta(n)$ by its mixed Gaussian limit — is far from $\theta(n)$.

# A Asymptotic inversion

The goal of the section is proving Theorem 1. The proof of Theorem 2 is similar, yet simpler as there is no need to control the remainder.

In the entire section Hypothesis 1 is on hold.
<figure><img src="image_4.png" /><figcaption>Figure 2: Kolmogorov-Smirnov distances in log-log scales between the renormalized score, its derivatives up to order $k$ and their limits, as well as the MLE and its asymptotic limit. The dashed line has a slope $-1/2$.</figcaption></figure>
## A.1 A Taylor formula with remainder

In this section, we basically consider the p-th order Taylor polynomial of $z \mapsto F(n, \theta_0 + \sum_{k=0}^p \alpha_k z^k)$ around 0 and a convenient expression for the remainder.

**Hypothesis 5.** Let $p \in \mathbb{N} \setminus \{0\}$. Let $\alpha := \{\alpha_k\}_{k=0, \dots, p} \in \mathbb{R}^{p+1}$ with $\alpha_0 = 0$. With $P_m$ defined by

$$P_m(z; \boldsymbol{\alpha}) := \sum_{k=0}^{m} \alpha_k z^k \quad \text{for } m = 0, 1, \dots, p \text{ and } z \in \mathbb{R},$$

we assume that $\theta_0 + P_m((0, 1); \boldsymbol{\alpha}) \subseteq \Theta$ for all $m \in \{1, \dots, p\}$.

For the sake of simplicity, in this section, we consider a real-valued function $F$ on $\Theta$, but we could consider a real-valued function on $\mathbb{N} \times \Theta$.

**Notation 5.** For $m = 1, \dots, p$, $z \in \mathbb{R}$, and $F \in \mathcal{C}^p(\Theta, \mathbb{R})$ we define

$$R_m(z; \boldsymbol{\alpha}) := \sum_{k_1+\cdots+k_m \le p} \alpha_{k_1} \cdots \alpha_{k_m} z^{k_1+\cdots+k_m} \\ \times \int_0^1 \int_0^{\tau_1} \cdots \int_0^{\tau_{m-1}} \left( \mathrm{D}^m F(\theta_0 + \tau_m P_{p-(k_1+\cdots+k_{m-1})}(z; \boldsymbol{\alpha})) \right. \\ \left. - \mathrm{D}^m F(\theta_0 + \tau_m P_{p-(k_1+\cdots+k_m)}(z; \boldsymbol{\alpha})) \right) \mathrm{d}\tau_m \cdots \mathrm{d}\tau_1.$$

**Notation 6.** We define for $z \in (0, 1)$,

$$P_p^*(z; \boldsymbol{\alpha}) := \sum_{k=1}^{p} |\alpha_k| \cdot |z|^k,$$

$$J(z; \boldsymbol{\alpha}) := [\theta_0 - P_p^*(z; \boldsymbol{\alpha}), \theta_0 + P_p^*(z; \boldsymbol{\alpha})] \cap \Theta,$$

$$\text{and } |F|_m(z; \boldsymbol{\alpha}) := \sup_{w \in J(z; \boldsymbol{\alpha})} |\mathrm{D}^m F(w)|.$$

**Lemma 1.** Assume Hypothesis 5. Let $F \in \mathcal{C}^{p+1}(\Theta, \mathbb{R})$. Then for all $z \in (0, 1)$,

$$F(\theta_0 + P_p(z; \boldsymbol{\alpha})) - F(\theta_0) \\ = \sum_{m=1}^{p} \frac{1}{m!} \mathrm{D}^m F(\theta_0) \sum_{k_1+\cdots+k_m \le p} \alpha_{k_1} \cdots \alpha_{k_m} z^{k_1+\cdots+k_m} + \sum_{m=1}^{p} R_m(z; \boldsymbol{\alpha}). \quad (17)$$

Besides, for $m = 1, \dots, p$,

$$|R_m(z; \boldsymbol{\alpha})| \le |F|_{m+1}(z; \boldsymbol{\alpha}) P_p^*(1; \boldsymbol{\alpha})^{m+1} |z|^{p+1} \quad \text{for } |z| \le 1. \quad (18)$$
*Proof.* For simplicity we do not specify the dependence on the sequence $\alpha$. By the fundamental theorem of calculus,

$$ 
\begin{aligned}
F(\theta_0 + P_p(z)) - F(\theta_0) &= \int_0^1 DF(\theta_0 + \tau P_p(z)) \cdot P_p(z) \, d\tau \\
&= DF(\theta_0)P_p(z) + \int_0^1 (DF(\theta_0 + \tau_1 P_p(z)) - DF(\theta_0)) \cdot P_p(z) \, d\tau_1.
\end{aligned} 
\quad (19) $$

Adding and removing another suitable term yields

$$ 
\begin{aligned}
F(\theta_0 + P_p(z)) - F(\theta_0) &= DF(\theta_0)P_p(z) + \sum_{k_1=1}^{p} \alpha_{k_1} z^{k_1} \int_0^1 (DF(\theta_0 + \tau_1 P_{p-k_1}(z)) - DF(\theta_0)) \, d\tau_1 \\
&\quad + \sum_{k_1=1}^{p} \alpha_{k_1} z^{k_1} \int_0^1 (DF(\theta_0 + \tau_1 P_p(z)) - DF(\theta_0 + \tau_1 P_{p-k_1}(z))) \, d\tau_1.
\end{aligned} 
\quad (20) $$

Note that $P_{p-k_1}(z) \in J(z; \alpha)$ for any $k_1 \in \{1, \dots, p\}$ with $J(z; \alpha)$ defined in Notation 6. The Lipschitz constant of $DF$ on $J(z; \alpha)$ is the sup-norm of $D^2 F$ on $J(z; \alpha)$, that is $|F|_2(z)$ in Notation 6. Then

$$ 
\begin{aligned}
& \sum_{k_1=1}^{p} |\alpha_{k_1}| \cdot |z|^{k_1} \cdot |DF(\theta_0 + \tau_1 P_p(z)) - DF(\theta_0 + \tau_1 P_{p-k_1}(z))| \\
& \le |F|_2(z) \sum_{k_1=1}^{p} \sum_{k_2=p-k_1+1}^{p} |\alpha_{k_1} \alpha_{k_2}| \cdot |z|^{k_1+k_2} \le |F|_2(z) \sum_{\substack{1 \le k_1, k_2 \le p \\ k_1+k_2 \ge p+1}} |\alpha_{k_1} \alpha_{k_2}| \cdot |z|^{k_1+k_2}.
\end{aligned} 
$$

This justifies our definition of $R_1(z)$ in Notation 5 as $|R_1(z)| = O(|z|^{p+1})$ for $z$ close to 0. Besides,

$$ \sum_{\substack{1 \le k_1, k_2 \le p \\ k_1+k_2 \ge p+1}} |\alpha_{k_1} \alpha_{k_2}| \le \left( \sum_{k=1}^{p} |\alpha_k| \right)^2 \le P_p^*(1)^2. $$

This proves (18) for $R_1$.

We could then iterate the above decomposition in (20) by replacing $F$ by $DF$ for
the second term in the right-hand side of the above integral. We obtain

$$ 
\begin{aligned}
DF(\theta_0 + \tau_1 P_{p-k_1}(z)) - DF(\theta_0) ={}& \tau_1 D^2 F(\theta_0) P_{p-k_1}(z) \\
& + \sum_{k_2 \le p-k_1} \tau_1 \alpha_{k_2} z^{k_2} \int_0^1 \left( D^2 F(\theta_0 + \tau_2 \tau_1 P_{p-k_1-k_2}(z)) - D^2 F(\theta_0) \right) d\tau_2 \\
& + \sum_{k_2 \le p-k_1} \tau_1 \alpha_{k_2} z^{k_2} \int_0^1 \left( D^2 F(\theta_0 + \tau_1 \tau_2 P_{p-k_1}(z)) - D^2 F(\theta_0 + \tau_2 \tau_1 P_{p-k_1-k_2}(z)) \right) d\tau_2.
\end{aligned} 
\quad (21) $$

Using a change of variable $\tau'_2 = \tau_1 \tau_2$ allows one to transform $\tau_1 \int_0^1 f(\tau_1 \tau_2) d\tau_2$ to $\int_0^{\tau_1} f(\tau_2) d\tau_2$ for any suitable function $f$.

Integrating (21) between 0 and 1 and using $p$ times the development (20) to the successive derivatives of $F$ leads to (17).

The remainder's terms $R_k$ are controlled in a similar way as for $R_1$.

□

## A.2 Application of the Taylor formula

In this section, we use Notations 2, 3, 5-6.

**Proposition 2.** Assume Hypothesis 2, in particular $\beta = \beta^*$ with $\bullet \in \{\tau, \tilde{\tau}\}$. Let $n \in \mathbb{N}$ such that $F(n, \cdot) \in \mathcal{C}^{p+1}(\Theta, \mathbb{R})$ and $\{F\}_1(n) \neq 0$. Let $\alpha(n) := \alpha^*(\delta^F(n)) \in \mathbb{R}^{p+1}$ and assume that $\theta_0 + P_m(\varphi(n); \alpha(n)) \in \Theta$ for all $m \in \{1, \dots, p\}$. Then

$$ 
\begin{aligned}
|F(n, \theta_0 + P_p(\varphi(n); \alpha(n)))| \le & \sum_{m \in \{2, \dots, p+1\}} P_p^*(1; \alpha(n))^m |F|_m(\varphi(n); \alpha(n)) |\varphi(n)|^{p+1} \\
& + \sum_{\substack{m \in \{1, \dots, p\} \\ \text{even}, \bullet = \tilde{\tau}}} P_p^*(1; \alpha(n))^m |\{F\}_m(n)| \cdot |\varphi(n)|^{p-1}.
\end{aligned} 
$$

*Proof.* In the proof we replace any dependence on $(\varphi(n); \alpha(n))$ by simply $(s)$. Using Notation 2, we rewrite (17) in Lemma 1 as

$$ F(n, \theta_0 + P_p(s)) - \{F\}_0(n)\varphi(n)^{-1} = \sum_{m=1}^{p} \frac{\{F\}_m(n)}{m!} \sum_{k_1+\cdots+k_m \le p} \alpha_{k_1}(n) \cdots \alpha_{k_m}(n) \varphi(n)^{k_1+\cdots+k_m-\beta_m} + \sum_{m=1}^{p} R_m(s). $$

For $m = 1, \dots, p$, the control (18) implies that

$$ |R_m(s)| \le |F|_{m+1}(s) P_p^*(1; \alpha(n))^{m+1} |\varphi(n)|^{p+1} \quad \text{since } |\varphi(n)| \le 1 $$
for all $m \in \{1, \dots, p\}$. We split

$$ \begin{aligned} & \{(k_1, \dots, k_m) \mid 1 \le k_1 + \dots + k_m \le p\} \\ &= \{(k_1, \dots, k_m) \mid 1 \le k_1 + \dots + k_m < p - 1 + \beta_m\} \\ & \quad \cup \{(k_1, \dots, k_m) \mid p - 1 + \beta_m \le k_1 + \dots + k_m \le p\}. \end{aligned} $$

Note that the latter set is empty when $p-1+\beta_m = p+1$, that is when $\beta_m = 2$. It is not empty when $\beta = \beta^\wedge$, $p \ge 2$, for all even $m$ and it holds that

$$ \left| \sum_{k_1+\dots+k_m=p} \alpha_{k_1}(n) \cdots \alpha_{k_m}(n) |\varphi(n)|^{k_1+\dots+k_m} \right| \le \sum_{k_1+\dots+k_m=p} |\alpha_{k_1}(n) \cdots \alpha_{k_m}(n)| \cdot |\varphi(n)|^p \le P_p^*(1; \alpha(n))^m |\varphi(n)|^p. $$

Using (11) we obtain the result.

<figure><img src="image_5.png" /></figure>

## A.3 Proof of Theorem 1 on the approximation of roots

*Proof of Theorem 1.* Since $DF(n, \cdot)$ is continuous on $U(n)$, The fact that $|DF(n, \cdot)|$ has a positive lower bound on $U(n)$ (Condition (ii)) implies that $F(n, \cdot)$ is invertible on $U(n)$ by the Inverse Function Theorem [1]. Besides, for any $\sigma, \sigma' \in F(n, U(n))$,

$$ \begin{aligned} |DF^{-1}(n, \sigma) - DF^{-1}(n, \sigma')| &\le \sup_{r \in [\sigma, \sigma')} \frac{|\sigma - \sigma'|}{|DF(n, F^{-1}(n, \tau))|} \\ &\le \sup_{\theta \in U(n)} \frac{|\sigma - \sigma'|}{|DF(n, \theta)|} = \sup_{\theta \in U(n)} \frac{|\varphi(n)|^{\beta_1} |DF(n, \theta)|}{|\varphi(n)|^{\beta_1} |\varphi(n)|} |\sigma - \sigma'| \le \frac{|\varphi(n)|^{\beta_1}}{c(n)} |\sigma - \sigma'|. \end{aligned} $$

Condition (iii) implies that $J(\varphi(n)) \subset U(n)$, where $J$ is defined in Notation 6.

With $g(n) := F(n, \theta_0 + P_p(\varphi(n); \alpha(n)))$ and since $F(n, \theta(n)) = 0$,

$$ |\theta_0 + P_p(\varphi(n); \alpha(n)) - \theta(n)| = |F^{-1}(n, g(n)) - F^{-1}(n, 0)| \le \frac{|\varphi(n)|^{\beta_1}}{c(n)} |g(n)|. $$

The result follows from Proposition 2.

<figure><img src="image_6.png" /></figure>

## References

[1] R. Abraham, J. E. Marsden, and T. Ratiu. *Manifolds, tensor analysis, and applications*. Second. Vol. 75. Applied Mathematical Sciences. Springer-Verlag, New York, 1988. DOI: 10.1007/978-1-4612-1029-0.
[2] A. Anastasiou and C. Ley. "Bounds for the asymptotic normality of the maximum likelihood estimator using the Delta method". In: *ALEA, Lat. Am. J. Probab. Math. Stat.* 14.1 (2017), p. 153. DOI: 10.30757/alea.v14-09.

[3] A. Anastasiou and G. Reinert. "Bounds for the normal approximation of the maximum likelihood estimator". In: *Bernoulli* 23.1 (Feb. 2017). DOI: 10.3150/15-bej741.

[4] F. J. Anscombe. "Normal likelihood functions". In: *Ann. Inst. Stat. Math.* 16 (1964), pp. 1-19. DOI: 10.1007/BF02868558.

[5] O. Barndorff-Nielsen. "On a formula for the distribution of the maximum likelihood estimator". In: *Biometrika* 70.2 (1983), pp. 343-365. DOI: 10.1093/biomet/70.2.343.

[6] M. S. Bartlett. "Approximate confidence intervals". In: *Biometrika* 40 (1953), pp. 12-19. DOI: 10.1093/biomet/40.1-2.12.

[7] A. Bensoussan, J.-L. Lions, and G. Papanicolaou. *Asymptotic analysis for periodic structures*. Providence, RI: AMS Chelsea Publishing, 2011.

[8] J. P. N. Bishwal. "Berry-Esseen inequalities for the discretely observed Ornstein-Uhlenbeck-gamma process". In: *Markov Process. Related Fields* 17.1 (2011), pp. 119-150.

[9] L. D. Brown. *Fundamentals of statistical exponential families with applications in statistical decision theory*. English. Vol. 9. IMS Lect. Notes, Monogr. Ser. Hayward, CA: Institute of Mathematical Statistics, 1986.

[10] G. M. Cordeiro and F. Cribari-Neto. *An introduction to Bartlett correction and bias reduction*. SpringerBriefs in Statistics. Springer, Heidelberg, 2014. DOI: 10.1007/978-3-642-55255-7.

[11] R. B. Dingle. *Asymptotic expansions: Their derivation and interpretation*. London-New York: Academic Press, a subsidiary of Harcourt Brace Jovanovich Publishers. XV. 1973.

[12] R. J. M. M. Does, L. W. G. Strijbosch, and W. Albers. "Using Jackknife Methods for Estimating the Parameter in Dilution Series". In: *Biometrics* 44.4 (Dec. 1988), p. 1093. DOI: 10.2307/2531738.

[13] H. Döring, S. Jansen, and K. Schubert. "The method of cumulants for the normal approximation". English. In: *Probab. Surv.* 19 (2022), pp. 185-270. DOI: 10.1214/22-PS7.

[14] B. Efron. "Bootstrap confidence intervals for a class of parametric problems". In: *Biometrika* 72.1 (1985), pp. 45-58. DOI: 10.1093/biomet/72.1.45.
[15] B. Efron and D. V. Hinkley. "Assessing the accuracy of the maximum likelihood estimator: observed versus expected Fisher information". In: *Biometrika* 65 (1978), pp. 457-487. DOI: 10.1093/biomet/65.3.457.

[16] H. Furstenberg. "Algebraic functions over finite fields". In: *J. Algebra* 7.2 (Oct. 1967), pp. 271-277. DOI: 10.1016/0021-8693(67)90061-0.

[17] I. M. Gessel. "Lagrange inversion". In: *J. Combin. Theory Ser. A* 144 (2016), pp. 212-249. DOI: 10.1016/j.jcta.2016.06.018.

[18] R. D. Gill. "Non- and semi-parametric maximum likelihood estimators and the von Mises method. I". In: *Scand. J. Statist.* 16.2 (1989). With a discussion by J. A. Wellner and J. Præstgaard and a reply by the author, pp. 97-128.

[19] R. D. Gill and A. W. van der Vaart. "Non- and semi-parametric maximum likelihood estimators and the von Mises method. II". In: *Scand. J. Statist.* 20.4 (1993), pp. 271-288.

[20] A. Hall. *Generalized Method of Moments*. Oxford University Press, 2005.

[21] J. Hurt. "Asymptotic expansions of functions of statistics". English. In: *Apl. Mat.* 21 (1976), pp. 444-456.

[22] W. P. Johnson. "The curious history of Faà di Bruno's formula". In: *Am. Math. Mon.* 109.3 (2002), pp. 217-234. DOI: 10.2307/2695352.

[23] O. J. Karst and H. Polowy. "Sampling properties of the median of a Laplace distribution". In: *Amer. Math. Monthly* 70 (1963), pp. 628-636. DOI: 10.2307/2311631.

[24] P. A. Lagerstrom. *Matched asymptotic expansions. Ideas and techniques*. Vol. 76. New York etc.: Springer-Verlag, 1988.

[25] A. Lejay and S. Mazzonetto. "Maximum likelihood estimator for skew Brownian motion: The convergence rate". In: *Scand. J. Stat.* (Nov. 2023). DOI: 10.1111/sjos.12694.

[26] M. N. Mishra and B. L. S. Prakasa Rao. "On a Berry-Esseen type bound for the maximum likelihood estimator of a parameter for some stochastic partial differential equations". In: *J. Appl. Math. Stoch. Anal.* 2 (2004), pp. 109-122. DOI: 10.1155/S1048953304309019.

[27] W. K. Newey and R. J. Smith. "Higher Order Properties of GMM and Generalized Empirical Likelihood Estimators". In: *Econometrica* 72.1 (2004), pp. 219-255. URL: http://www.jstor.org/stable/3598854 (visited on 06/17/2022).

[28] Y. Pawitan. *In all likelihood: statistical modelling and inference using likelihood*. Oxford University Press, 2001.
[29] I. M. Skovgaard. "Edgeworth expansions of the distributions of maximum likelihood estimators in the general (non-i.i.d.) case". In: *Scand. J. Statist.* 8.4 (1981), pp. 227-236.

[30] A. D. Sokal. "A ridiculously simple and explicit implicit function theorem". In: *Sém. Lothar. Combin.* 61A (2009/11), Art. B61Ad, 21.

[31] D. A. Sprott. "Application of maximum likelihood methods to finite samples". In: *Sankhyā Ser. B* 37.3 (1975), pp. 259-270.

[32] D. A. Sprott. "Normal likelihoods and their relation to large sample theory of estimation". In: *Biometrika* 60 (1973), pp. 457-465. DOI: 10.1093/biomet/60.3.457.

[33] D. A. Sprott and R. Viveros-Aguilera. "The interpretation of maximum-likelihood estimation". In: *Can. J. Stat.* 12 (1984), pp. 27-38. DOI: 10.2307/3314722.

[34] A. W. van der Vaart. *Asymptotic statistics*. Vol. 3. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge, 1998. DOI: 10.1017/CBO9780511802256.

[35] B. Wang, H. Jiang, and J. Yu. "Berry-Esseen bound for parameter estimation in some time inhomogeneous diffusions and applications". In: *Statist. Probab. Lett.* 81.8 (2011), pp. 921-929. DOI: 10.1016/j.spl.2011.04.002.
# Supplement: Beyond the delta Method

This supplement not only revisits the main results of the paper—introducing greater flexibility—but also provides additional insights into the expansions and the properties of their coefficients.

In the first section, we present a more general formulation of our main results: the asymptotic implicit function theorems, which yields (pathwise) expansions of the estimator. A key feature of this formulation is the decoupling of the scale (typically representing sample size) and the rate (a function of the scale), treating the rate as a variable. This added flexibility is particularly useful for applications involving uncountable or multi-dimensional scales and was instrumental in identifying the coefficients of the estimator's expansion.

Section S2 focuses on the expansion's coefficients: their derivation, the issue of non-uniqueness, and alternative but equivalent formulations in specific cases, namely “up-flat” and “zig-zag”. We also explore how the expansion is affected by perturbations of the zeros or score functions (see Section S2.8) and provide an illustrative application in Example 7 in Section S4.

In Section S3, we analyze the effects of a change of variables in the parameter space—specifically, reparametrization. We show that replacing $F(n, \cdot)$ with $F(n, \cdot)/\Phi$ for a suitable $\Phi$ does not alter the root $\theta(n)$ but may yield a different expansion in Eq. (1), which can be more convenient in practice.

Finally, in Section S4, we present an additional statistical example involving diffusion processes, different from Skew-BM. In this case, observations are taken over a long time horizon with a fixed time-lag, as opposed to the high-frequency setting of Skew-BM.

# S1 The asymptotic implicit function theorem

## S1.1 The deterministic asymptotic implicit function theorem

Let us introduce the subsets $\Theta \subset \mathbb{R}$ (the space of parameters), $I \subset \mathbb{R}$, the space of rates, and $S$, the space of scales.

*Hypothesis 6.* The space $\Theta$ contains an open interval around 0, $(0, \epsilon) \subset I$ for some $\epsilon > 0$, and there exists a function $\varphi : S \to I$, which we call the *rate function*.

*Hypothesis 7.* We fix $p \in \{1, \dots, +\infty\}$ and we consider $\boldsymbol{\beta} := \{\beta_k\}_{k=0,1,\dots,p+1} \in \mathbb{Z}^{p+2}$.
**Notation 7.** For a parametric family of functions $\theta \mapsto F(s, \theta)$ which is of class $\mathcal{C}^p(\Theta, \mathbb{R})$ for any $s \in \mathbf{S}$ (or $F \in \mathcal{C}^p(\Theta, \mathbb{R})^{\mathbf{S}}$), we denote by $\mathbf{D}^m$ the derivative of order $m$ with respect to $\theta$. We define

$$ \{F\}_m(s) := \varphi(s)^{\beta_m} \mathbf{D}^m F(s, 0) \text{ for } m = 0, \dots, p \text{ and } s \in \mathbf{S}. $$

Let us introduce two notations, one related to $\beta$, and the other related to multi-indices.

**Notation 8.** We define

$$ \beta_{\star} := \max_{m=2,\dots,p+1} \beta_m, \qquad (S1) $$

$$ \gamma_m := p + 1 + (\beta_m - \beta_{\star}) \wedge 0 \text{ for } m = 1, \dots, p. \qquad (S2) $$

**Notation 9.** In a systematic manner, we use sums overs multi-indices satisfying some constraints. Such multi-indices are denoted by $(k_1, \dots, k_m)$ and each $k_i$ is an integer.

We state a series expansion which approximates the roots of $F(s, \cdot)$.

**Theorem 4 (Approximation of roots).** Assume Hypotheses 6 and 7 hold and that $\beta_1 \ge \beta_{\star}$. Let $F \in \mathcal{C}^{p+1}(\Theta, \mathbb{R})^{\mathbf{S}}$. Let $s \in \mathbf{S}$ such that there exist a sequence $\alpha(s) = \{\alpha_k(s)\}_{k=0,\dots,p}$ and a convex neighborhood $U(s)$ of 0 in $\Theta$ satisfying:

(i) $\alpha_0(s) = 0$ and

$$ \{F\}_0(s)\varphi(s)^{-\beta_0} + \sum_{m=1}^{p} \frac{\{F\}_m(s)}{m!} \sum_{k_1+\dots+k_m < \gamma_m} \alpha_{k_1}(s) \cdots \alpha_{k_m}(s) \varphi(s)^{\sum_{n=1}^{m} k_n - \beta_m} = 0. \quad (S3) $$

(ii) There exists $\theta(s) \in U(s)$ such that $F(s, \theta(s)) = 0$.

(iii) It holds that

$$ |\varphi(s)|^{\beta_1} |DF(s, w)| \ge c(s) > 0, \text{ for all } w \in U(s) \\ \text{and } \max_{m \in \{2, \dots, p+1\} \ w \in U(s)} \sup_{w \in U(s)} |\mathbf{D}^m F(s, w)| < +\infty. $$

(iv) It holds that $\sum_{i=1}^p |\alpha_i(s)| \cdot |\varphi(s)|^i \in U(s)$.

Then

$$ \left| \theta(s) - \sum_{i=1}^{p} \alpha_i(s) \varphi(s)^i \right| \le K(s) \max \left\{ \sum_{k=1}^{p} |\alpha_k(s)|, \left( \sum_{k=1}^{p} |\alpha_k(s)| \right)^{p+1} \right\} |\varphi(s)|^{p+1}, $$
with

$$K(s) \le \frac{\max_{m \in \{1, \dots, p\}} |\varphi(s)|^{\beta_1 - \beta_*} |\{F\}_m(s)| + \max_{m \in \{2, \dots, p+1\}} \sup_{w \in U(s)} |\varphi(s)|^{\beta_1} |\mathbb{D}^m F(s, w)|}{\beta_{m+1} \le \beta_*} < \infty.$$

The proof of Theorem 4 is similar to the one of Theorem 4, provided in Appendix A. It is thus omitted. We also consider the case of analytic functions, in which the expansion is infinite.

**Theorem 5 (Approximation of roots, analytic case).** Assume that Hypotheses 6 and 7 hold for $p = +\infty$. Let $F: S \times \Theta \to \mathbb{R}$ be such that $F(s, \cdot)$ is analytic. Let $s \in S$ such that there exist a sequence $\alpha(s) = \{\alpha_k(s)\}_{k \ge 0}$ and a convex neighborhood $U(s)$ of 0 in $\Theta$ satisfying:

(i) $\alpha_0(s) = 0$ and (S3) holds with $p = +\infty$ and $\gamma_m = +\infty$ for $m \ge 1$.

(ii) There exists $\theta(s) \in U(s)$ such that $F(s, \theta(s)) = 0$.

(iii) It holds that $|\varphi(s)|^{\beta_1} |\mathbb{D}F(s, w)| \ge c(s) > 0$ for all $w \in U(s)$.

(iv) It holds that $\sum_{i=1}^{+\infty} |\alpha_i(s)| \cdot |\varphi(s)|^i \in U(s)$.

Then

$$\theta(s) = \sum_{i=1}^{+\infty} \alpha_i(s) \varphi(s)^i.$$

In the next sections, in particular in Section S2 we define the notion of $(\beta, p)$-related sequence to a given sequence $\delta$ and we show that, if $\alpha$ is a $(\beta, p)$-related sequence to $\delta$, then it satisfies an equation analogous to (S3).

$$\delta_0 \varphi(s)^{-\beta_0} + \sum_{m=1}^{p} \delta_m \sum_{k_1 + \cdots + k_m < \gamma_m} \alpha_{k_1} \cdots \alpha_{k_m} \varphi(s)^{\sum_{n=1}^{m} k_n - \beta_m} = 0. \quad (S4)$$

Note that (S3) corresponds to (S4) with $\delta(s) = \{\{F\}_k(s)/k!\}_{k=0,\dots,p}$, but also with $\delta(s) = \{-\{F\}_k(s)/(k!\{F\}_1(s))\}_{k=0,\dots,p}$ when $\{F\}_1(s) \ne 0$.

Later, we focus on two sequences

$$\beta^{-} := (1, 2, 2, \dots, 2) \text{ and } \beta^{+} := (1, 2, 1, 2, \dots, 1, 2)$$

which are relevant for statistical applications. We refer to these choices of $\beta$ respectively as “up-flat” and “zigzag” cases.

For the reader convenience, we provide sequences $\alpha$ which are $(\beta, p)$-related sequence to a given sequence $\delta$ with $\delta_1 = -1$ and $\beta = \beta^{-}$ or $\beta^{+}$. They were already given in (9) and (10), we recall them here. For $\beta^{-}$,

$$\alpha_1 = \delta_0 \quad \text{and} \quad \alpha_q = \sum_{m=2}^{q} \delta_m \sum_{k_1 + \cdots + k_m = q} \alpha_{k_1} \cdots \alpha_{k_m} \quad \text{for } 2 \le q \le p, \quad (S5)$$
and, for $\beta^{\sim}$, take $\alpha_1 = \delta_0$, $\alpha_{2q} = 0$ for $2q \le p$ and for $2q + 1 \le p$

$$ \alpha_{2q+1} = \sum_{\substack{m=2 \\ m \text{ even}}}^{2q} \delta_m \sum_{k_1+\cdots+k_m=2q} \alpha_{k_1} \cdots \alpha_{k_m} + \sum_{\substack{m=3 \\ m \text{ odd}}}^{2q+1} \delta_m \sum_{k_1+\cdots+k_m=2q+1} \alpha_{k_1} \cdots \alpha_{k_m}. \quad (S6) $$

We refer to Lemmas 5, 6 and 8 in Section S2 for the proofs.

## S1.2 The stochastic asymptotic implicit function theorem

In this section we consider the inference of a parameter $\theta_0 \in \mathbb{R}$ and we assume that $\Theta$ contains an open interval around $\theta_0$.

To deal with convergence, we need some topological assumptions. We consider that $(S, \prec)$ is a directed set that is a set with a partial order $\prec$ such that for any $s, s'$ in $S$, there exists $r \in S$ such that $s \prec r$ and $s' \prec r$. Typically, we consider $S = \mathbb{R}$ or $S = \mathbb{N}$ (sample size) yet this framework allows consider partitions and pairs (sample size and discretization step).

**Hypothesis 8.** The rate function $\varphi: S \to \mathbb{I} \cap (0, 1]$ is monotone decreasing, that is $\varphi(s) \le \varphi(s')$ whenever $s' \prec s$.

Note that, since $(S, \prec)$ is a directed set, the rate function is a net.

**Notation 10.** Let $F \in \mathcal{C}^p(\Theta, \mathbb{R})^S$ such that $DF(s, \theta_0) \ne 0$ then, analogously to Notation 7, let for all $k \in \{0, \dots, p\}$,

$$ \{F\}_k(s) := \varphi(s)^{\beta_k} \mathbb{D}^k F(s, \theta_0). $$

Let $\delta^F$ be the family of $(\mathbb{R}^{p+1})^S$ defined by

$$ \delta^F(s) := \left\{ -\frac{1}{k!} \frac{\{F\}_k(s)}{\{F\}_1(s)} \right\}_{k=0,\dots,p}. $$

In the next result, our main result, we use the notion of $(\beta, p)$-related sequence to a given sequence $\delta$. This notion has been briefly presented in the previous section, but it is introduced in full details in Section S2.

**Proposition 3.** We consider a family of random functions $(F(s, \theta))_{(s,\theta) \in S \times \Theta}$ on a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ as well as $p$ as in Hypothesis 7. Assume that for all $s \in S$ and $\mathbb{P}$-almost all $\omega \in \Omega$ the function $\theta \mapsto F(s, \theta)$ is of class $\mathcal{C}^p$ at $\theta_0$, and there exists a random vector $\mu = \{\mu_k\}_{k=0}^p$ such that

$$ (\{F\}_0(s), \dots, \{F\}_p(s)) \xrightarrow{\text{dist.}} \mu. $$

For every $s \in S$ let $\alpha(s)$ the $(\beta, p)$-related sequence to $\delta^F(s)$. Then
$$ (i) \quad \delta^F(s) \xrightarrow{\text{dist.}} \delta := \left\{ -\frac{1}{m!} \frac{\mu_m}{\mu_1} \right\}_{m=0,\dots,p}. $$

$$ (ii) \quad \alpha(s) \xrightarrow{\text{dist.}} \alpha \text{ where } \alpha \text{ is the } (\beta, p)-\text{related sequence to } \delta. $$

**Definition 2 (Boundedness in probability).** A family of random variables $\{X_s\}_{s \in \mathbf{S}}$ is said to be *bounded in probability* if for any $\epsilon > 0$, there exist $K \ge 0$ and $s \in \mathbf{S}$ such that $\sup_{s<\cdot s'} \mathbb{P}[|X_{s'}| \ge K] \le \epsilon$.

**Hypothesis 9.** We consider a family of random functions $(F(s, \theta))_{(s,\theta)\in\mathbf{S}\times\Theta}$ on a probability space $(\Omega, \mathcal{F}, \mathbb{P})$ as well as $\beta \in \{\beta^{-}, \beta^{\sim}\}$ and $p$ as in Hypothesis 7 and the rate function as in Hypothesis 8. Assume that there exists an open neighborhood $U \subseteq \Theta$ of $\theta_0$ such that for all $s \in \mathbf{S}$ and $\mathbb{P}$-almost all $\omega \in \Omega$:

$$ (i) \quad \theta \mapsto F(s, \theta) \text{ is of class } \mathcal{C}^{p+1} \text{ in } U. $$

$$ (ii) \quad \text{There exists } \theta(s) \in U \text{ such that } F(s, \theta(s)) = 0. $$

$$ (iii) \quad DF(s, \theta) \neq 0 \text{ for all } \theta \in U \setminus \{\theta_0\} \text{ and } \{1/|F|_1(r)\}_{r \in \mathbf{S}} \text{ is bounded in probability.} $$

$$ (iv) \quad \text{The family } \{N_r(F)\}_{r \in \mathbf{S}} \text{ is bounded in probability with} $$

$$ N_s(F) := \sup_{1 \le k \le p+1} \sup_{\theta \in U} |\varphi(s)^{\beta_1} D^k F(s, \theta)|. $$

The following proposition is a direct consequences of Theorem 4.

**Theorem 6.** Assume Hypothesis 9 and assume that the net $\varphi$ in Hypothesis 8 converges to 0. Assume that there exists a random vector $\mu = \{\mu_k\}_{k=0}^p$ such that

$$ (\{F\}_0(s), \dots, \{F\}_p(s)) \xrightarrow{\text{dist.}} \mu, $$

and for every $s \in \mathbf{S}$ let $\alpha(s)$ the $(\beta, p)$-related sequence to $\delta^F(s)$ (see (S5)-(S6)). Let $\alpha$ and $\delta$ as in Proposition 3. We introduce

$$ P_{\ell}(\varphi(s); \alpha(s)) := \sum_{k=0}^{\ell} \alpha_k(s) \varphi(s)^k, \qquad (S7) $$

$$ \theta_{\ell}(s) := \theta_0 + P_{\ell}(\varphi(s), \alpha(s)) \quad \text{and} \quad \theta_{\ell}(s, \infty) := \theta_0 + P_{\ell}(\varphi(s), \alpha) \qquad (S8) $$

for $\ell = 0, \dots, p$. Then,

$$ (i) \quad P_p(\varphi(s); \alpha(s)) \text{ converges to } 0 \text{ in probability.} $$

$$ (ii) \quad |\theta(s) - \theta_0 - P_p(\varphi(s); \alpha(s))| \varphi(s)^{-(p+1)} \text{ is bounded in probability.} $$

$$ (iii) \quad \varphi(s)^{-(k+1)} (P_p(\varphi(s); \alpha(s)) - P_k(\varphi(s); \alpha(s))) \xrightarrow{\text{dist.}} \alpha_{k+1} \quad \text{and} $$

$$ \varphi(s)^{-(k+1)} (\theta(s) - \theta_k(s)) \xrightarrow{s \to \infty} \alpha_{k+1} $$

$$ \text{for } k \in \{0, \dots, p-1\} \text{ (with } P_0 = 0\text{). In particular if } k = 0, $$

$$ \varphi(s)^{-1}(\theta(s) - \theta) \xrightarrow{s \to \infty} \alpha_1. $$
(iv) $\theta(s) - \theta_p(s, \infty)$ converges to 0 in probability with rate $\varphi(s)$ ($\alpha_1(s) - \alpha_1$).

The previous result exhibits two approximations of $\theta(s)$, i.e.,

$$ \theta_p(s) := \theta_0 + P_p(\varphi(s), \boldsymbol{\alpha}(s)) \quad \text{and} \quad \theta_p(s, \infty) := \theta_0 + P_p(\varphi(s), \boldsymbol{\alpha}). $$

# S2 Formal manipulation of expansions

In Theorems 4 and 5, the expansion of the root $\theta(s)$ to $F(s, \theta(s)) = 0$ is performed in term of $\varphi(s)$ with $s$-dependent coefficients. This implies a lack of uniqueness in the expansion. To enforce uniqueness, we identify the coefficients by transforming series to formal ones. Although natural, this choice is arbitrary and “righteous”<fnref n="1" />.

The main idea is to decouple the scale ($s$, playing the role of sample size) and the rate ($\varphi(s)$), considering the latter as a variable. In Section S2.1 we introduce the necessary operators to decouple and re-couple: dummization and evaluation. Next, in Section S2.2 we recall Faà di Bruno and Taylor formulae and show how dummization and evaluation behave in Taylor expansion of composition of functions. In Section S2.3, we propose a definition of sequences that satisfy Eq. (S3). We call them $(\boldsymbol{\beta}, p)$-related sequences. We provide a different presentation in Section S2.4. Next we show that there exists a unique $(\boldsymbol{\beta}, p)$-related sequence when $\boldsymbol{\beta}$ is $\boldsymbol{\beta}^{-}$ and $\boldsymbol{\beta}^{\sim}$ and we provide the proof of the recursive formulas (S5)-(S6) in Sections S2.5-S2.6. In the “up-flat” case, we provide different characterizations of the $(\boldsymbol{\beta}^{-}, p)$-related sequence (see Lemmas 8 and 9) that we propose also in the context of Theorem 4 in Section S2.7. To conclude, we show in Section S2.8 how an additive perturbation acts on the sequence.

## S2.1 Dummization and evaluation

For a ring $\mathbb{K}$ and a family of indeterminates $\mathbf{X} := (X_1, \dots, X_n)$, $\mathbb{K}[X_1, \dots, X_n]$ (resp. $\mathbb{K}(X_1, \dots, X_n)$) denotes the ring of polynomials (resp. rational functions) with indeterminates $X_1, \dots, X_n$ and coefficients in $\mathbb{K}$. Also, the set of formal power series is $\mathbb{K}[[X_1, \dots, X_n]]$.

To a finite, numerical sequence $\mathbf{a} = (a_1, \dots, a_q)$, we consider the “dummization” operation which consists in transforming each of the $a_i$'s to an indeterminate $a_i^{\mathbf{q}}$. Therefore $\mathbb{R}[\mathbf{a}^{\mathbf{q}}]$ (resp. $\mathbb{R}(\mathbf{a}^{\mathbf{q}})$) is the ring of polynomials (resp. rational functions) whose indeterminates are the $a_i^{\mathbf{q}}$.

<footnote n="1">¹As pointed out in [2] regarding homogenization theory, a “criminal path” leads to other ways to compute effective coefficients.</footnote>
We consider another indeterminate $z$, that we will use later as a placeholder for $\varphi(s)^{\bullet}$ or for $x \in \mathbb{R}$. We denote by $\mathbb{K}[z]$ the ring of polynomials whose coefficients are in $\mathbb{K}$. We are concerned with $\mathbb{K} = \mathbb{R}(a^{\bullet})$. We also denote by $\mathbb{K}[z, z^{-1}]$ the Laurent series whose coefficients are in $\mathbb{K}$.

The evaluation — denoted by $e$ — consists in transforming back $a_i^{\bullet}$ to $a_i$. It is the dual operation of the dummization. We extend $e$ as an algebra homomorphism from $\mathbb{R}[[a^{\bullet}]]$ to $\mathbb{R}$. We furthermore extend $e$ on $\mathbb{R}(a^{\bullet})$ by setting $e(P/Q) := e(P)/e(Q)$ as well as to $\mathbb{K}[[z, z^{-1}]]$ by setting $e(z^k) = \varphi(s)^k$ for any $k \in \mathbb{Z}$.

**Notation 11.** Let $p \in \mathbb{N}$. Given a sequence $\delta = \{\delta_k\}_{k=0,...,p} \subset \mathbb{K}$, we denote by $P_p(z; \delta)$ the polynomial $P_p(z; \delta) := \sum_{k=0}^{p} \delta_k z^k \in \mathbb{K}[z]$.

**Notation 12** (Projection onto polynomials of degrees at most $p$). Let $p \in \mathbb{N}$. We denote by $\mathfrak{p}_p$ is the projection of a power series $P \in \mathbb{K}[[z]]$ onto the polynomials of degree at most $p$ by setting $z^q = 0$ whenever $q > p$.

Note that we can extend the latter notation to $p = \infty$ taking $\mathfrak{p}_{\infty}$ the identity and $P_{\infty}(z; \delta)$ the series $\sum_{k=0}^{\infty} \delta_k z^k$ when it is well defined.

The next lemma is immediate by expanding powers.

**Lemma 2.** Let $p \in \mathbb{N}$. Consider three families $\delta^{\bullet}$, $\alpha^{\bullet}$, $\eta^{\bullet}$ with components in a field $\mathbb{K}$. Assume that $\alpha_0^{\bullet} = 0$. These families $\delta^{\bullet}$, $\alpha^{\bullet}$ and $\eta^{\bullet}$ are linked by the relation

$$ \eta_0^{\bullet} = \delta_0^{\bullet} \quad \text{and} \quad \eta_k^{\bullet} = \sum_{m=1}^{k} \delta_m^{\bullet} \sum_{k_1+\cdots+k_m=k} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} \in \mathbb{R}(\delta^{\bullet}) \quad \text{for all } k \in \{1, \ldots, p\} \quad (S9) $$

if and only if

$$ P_p(z; \eta^{\bullet}) = \mathfrak{p}_p \circ P_p(\cdot; \delta^{\bullet}) \circ P_p(z; \alpha^{\bullet}). $$

## S2.2 Taylor and Faà di Bruno formulae

The Taylor formula is the main tool to go back and forth between the realm of analysis and the one of algebra.

The Faà di Bruno formula is a way to compute the higher order derivatives of the composition of two functions [7]. It is a useful tool to deal with power series [1, 8] and can be seen both from the points of view of analysis [9] or of algebra [5].
**Proposition 4** (Faà di Bruno formula). Let $U, V$ be open subsets of $\mathbb{R}$. Let $f: U \to \mathbb{R}$ and $g: V \to U$ be two functions of class $\mathscr{C}^p$ for $p \ge 1$. Then $f \circ g$ is of class $\mathscr{C}^p(V, \mathbb{R})$ and $2$

$$D^l(f \circ g)(x) = \ell! \sum_{m=1}^{\ell} \frac{D^m f(g(x))}{m!} \sum_{k_1+\dots+k_m=\ell} \frac{D^{k_1} g(x)}{k_1!} \dots \frac{D^{k_m} g(x)}{k_m!}, \quad (S10)$$

for $\ell = 1, \dots, p$ and $x \in V$.

**Definition 3** (Formal Taylor series). Let $f$ be a function of class $\mathscr{C}^p(\mathbb{R}, \mathbb{R})$ for $p = 1, \dots, +\infty$. We set $\delta_k := D^k f(0)/k!$, $k = 0, \dots, p$ and $\delta := \{\delta_k\}_{k=0,\dots,p}$. The Taylor series operator $\mathcal{T}_p$ is defined as

$$\mathcal{T}_p f(x) := P_p(x; \delta) = \sum_{k=0}^{p} \delta_k x^k \in \mathbb{R}[[x]]$$

that is, it gives the Taylor series of $f$ around 0. The formal Taylor series of $f$ is

$$\mathcal{T}_p^{\bullet} f(z) := P_p(z; \delta^{\bullet}) = \sum_{k=0}^{p} \delta_k^{\bullet} z^k \in \mathbb{K}[[z]] \text{ with } \mathbb{K} = \mathbb{R}[\delta^{\bullet}]. \quad (S11)$$

If $\epsilon(z) = x$ for $f \in \mathscr{C}^{p+1}(\mathbb{R}, \mathbb{R})$,

$$\epsilon(\mathcal{T}_p^{\bullet} f(z)) = \mathcal{T}_p f(x) = f(x) - R(x)$$

with $|R(x)| \le |x|^{p+1} \sup_{y \in [-x, x]} |D^{p+1} f(y)|/(p+1)!$ for any $x$ small enough.

A consequence of Lemma 2 and the Faà di Bruno formula is the following one.

**Lemma 3.** Let $p \in \mathbb{N}$, let $f, g$ and $h$ be three functions of class $\mathscr{C}^{p+1}$ such that $g(0) = 0$ and $\mathcal{T}_p k(x) = \epsilon(\mathcal{T}_p^{\bullet} k(z))$ for $k = f, g, h$ with $\epsilon(z) = x$.

Then $\mathcal{T}_p^{\bullet} h = \mathfrak{p}_p(\mathcal{T}_p^{\bullet} f \circ \mathcal{T}_p^{\bullet} g)$ if and only if $\mathcal{T}_p h = \mathcal{T}_p(f \circ g) = \mathfrak{p}_p(\mathcal{T}_p f \circ \mathcal{T}_p g)$.

We end this section by relating the composition of analytic functions and their formal developments.

**Notation 13.** For $r > 0$, we denote by $\mathcal{A}_r$ the class of analytic functions whose radius of convergence is at least $r$. We also set $B_r := \{x \in \mathbb{R} \mid |x| < r\}$ the open ball of radius $r$.

<footnote n="2">²We use a different convention than the usual ones on summing the indices written by regrouping the multi-index $(k_1, \dots, k_m)$ by their respective values and counting the number of such partitions.</footnote>
**Hypothesis 10.** The functions $f, g, h$ satisfy $f \in \mathcal{A}_{r_f}, g \in \mathcal{A}_{r_g}, h \in \mathcal{A}_{r_h}$ with $g(B_{r_g}) \subset B_{r_f}$, so that $f \circ g \in \mathcal{A}_{r_g}$.

Note that for an analytic function with a positive radius of convergence $r_f$, $f(x) = \mathcal{T}_\infty f(x)$ whenever $|x| < r_f$.

**Lemma 4.** Let the same assumptions of Lemma 3, and let $f, g, h$ as in Hypothesis 10. Then $\mathcal{T}_\infty h = \mathcal{T}_\infty f \circ \mathcal{T}_\infty g$ if and only if $h = f \circ g$.

**Remark 3.** Let $p \in \mathbb{N}$ and $\delta = \{\delta_k\}_{k=0}^\infty \subset \mathbb{R}$ such that $P_\infty(\cdot; \delta) \in \mathcal{A}_r$ for some $r > 0$. Then $P_p(x; \delta) = \mathfrak{p}_p P_\infty(x; \delta) = \mathcal{T}_p P_\infty(x; \delta)$ and $P_p(x; \delta) = \mathcal{T}_p P_p(x; \delta)$. In particular if $f \in \mathcal{A}_r$ for some $r > 0$, then $\mathcal{T}_p f = \mathfrak{p}_p \mathcal{T}_\infty f = \mathcal{T}_p \mathcal{T}_\infty f$.

## S2.3 Constructing $(\beta, p)$-related sequences

The goal of this section is, for $s \in \mathbb{S}$ fixed, to find a sequence $\alpha(s)$ satisfying (S3). The idea is to consider the coefficients $\alpha_m(s)^\bullet$ and $\{F\}_m(s)$ as dummy variables, then identify the coefficients $\alpha_m(s)^\bullet$ by the dummy version of Eq. (S3), and finally transfer them back to their numerical values.

**Definition 4** (($\beta, p$)-related sequences). Let $p$ and $\beta$ as in Hypothesis 7. Let $\delta = \{\delta_k\}_{k=0,...,p}$ and $\alpha = \{\alpha_k\}_{k=0,...,p}$. We say that $\alpha$ is $(\beta, p)$-related to $\delta$ if there exist $\alpha_k^\bullet, k = 0, ..., p$ which are rational functions in $\mathbb{K} := \mathbb{R}(\delta^\bullet)$ such that

(i) For $k = 0, ..., p$, $\alpha_k = \mathcal{E}(\alpha_k^\bullet)$ with $\alpha_0^\bullet = 0$.

(ii) In $\mathbb{K}[[z, z^{-1}]]$,

$$ \delta_0^\bullet z^{-\beta_0} + \sum_{m=1}^{p} \delta_m^\bullet \sum_{k_1 + \cdots + k_m < \gamma_m} \alpha_{k_1}^\bullet \cdots \alpha_{k_m}^\bullet z^{\sum_{n=1}^m k_n - \beta_m} = 0, \quad (S12) $$

with $\gamma_m$ defined by (S2), meaning that all the coefficients in $\mathbb{K}$ vanish for each of the powers of $z$.

**Remark 4.** If $\alpha$ is a $(\beta, p)$-related sequence to $\delta$, then $\alpha$ it is also $(\beta', p)$-related sequence to $\delta$ for $\beta' = \{\beta_k + \gamma\}_{k=0,...,p+1}$ for any $\gamma \in \mathbb{Z}$.

**Remark 5.** If $\alpha$ is a $(\beta, p)$-related sequence to $\delta$, then $\alpha$ it is also $(\beta, p)$-related sequence to $\delta_\lambda$ for $\delta_\lambda = \{\lambda \delta_k\}_{k=0,...,p}$ for any $\lambda \in \mathbb{R}$.

**Example 1** (Non-existence/non-uniqueness). We consider $p = 2, \beta_0 = 0, \beta_1 = 1, \beta_2 = 2$ and $\beta_3 \in \{1, 2\}$. Then $\gamma_m = 1 + \beta_m$ for $m = 1, 2$. Eq. (S12) becomes

$$ \delta_0^\bullet + \delta_1^\bullet \alpha_1^\bullet + \delta_2^\bullet (\alpha_1^\bullet)^2 = 0. $$

The possibility to find $\alpha_1^\bullet \in \mathbb{R}$ depends on the relative values of $\delta$. Even if we are able to do so, we are free to choose $\alpha_2^\bullet$ and the choice of $\alpha$ which is ((0, 1, 2), 2)-related to $\delta$ is not unique.
**Example 2** (Non-existence/non-uniqueness). We consider $p = 2$, $\beta = (0, 1, 1, 2)$. If $\delta_2^{\bullet} \neq 0$, then $\gamma_m = 1 + \beta_m$ for $m = 1, 2$. Eq. (S12) becomes

$$ \delta_0^{\bullet} + \delta_1^{\bullet} \alpha_1^{\bullet} + \delta_2^{\bullet} (\alpha_1^{\bullet})^2 z = 0, $$

so that this could not be solved unless $\delta_0^{\bullet} = 0$ and $\alpha_1^{\bullet} = 0$. In the latter case, we are free to choose $\alpha_2^{\bullet}$.

The next lemma relates the definition above to the assumption (i) in Theorem 4.

**Lemma 5.** Let $F \in \mathcal{C}^p(\Theta, \mathbb{R})^S$ such that $\{F\}_1 \neq 0$, and fix $s \in S$. Let $\alpha(s)$ be a $(\beta, p)$-related sequence to $\delta^F(s)$. Define the evaluation map so that $\epsilon(z) = \varphi(s)$. Then $\alpha(s)$ satisfies (S3).

We study below two examples of $\beta$ for which $\alpha$ is unique when using Definition 4. However, given a sequence $\alpha(s)$ satisfying (S3), it is possible to change $\alpha_1(s)$ by a term of order $\varphi(s)$ so that (S3) is still satisfied. Therefore, there are an infinite number of solutions to (S3). Our construction is a way to impose a unique choice of $\alpha(s)$ satisfying (S3) obtaining (S5) and (S6).

## S2.4 Scrambling operator

We provide here an alternative formulation of equation (S12) of Definition 4.

**Definition 5 (Scrambling operator).** Given $P_p(z; \delta) = \sum_{k=0}^p \delta_k z^k \in \mathbb{K}[[z]]$ and $\beta \in \mathbb{R}^{p+1}$, the scrambling operator is

$$ \mathfrak{s}_{\beta}(P_p(z; \delta)) := \sum_{k=0}^{p} \delta_k z^{k-\beta_k} \in \mathbb{K}[[z, z^{-1}]]. $$

With Definition 5, Notation 11 and Notation 12, (S12) is equivalent to

$$ \mathfrak{s}_{\beta} \circ \mathfrak{p}_p \circ P_p(\cdot; \delta^{\bullet}) \circ P_p(z; \alpha^{\bullet}) = 0 \quad \text{with} \quad \alpha_0^{\bullet} = 0. \qquad (S13) $$

Under the assumptions of Lemma 5, (S12) (and so (S13)) rewrites

$$ \mathfrak{s}_{\beta} \circ \mathfrak{p}_p \circ \mathcal{T}_p^{\bullet} F(s, P_p(z; \alpha(s)^{\bullet})) = 0 \quad \text{in} \quad \mathbb{R}(\delta^{\bullet}(s))[[z, z^{-1}]]. \qquad (S14) $$

We now study $(\beta, p)$-related sequences for some particular choices of $\beta$, which are fitted for statistical applications.
## S2.5 The case “zig-zag”

Our first case of interest is $\beta^{\sim} = (1, 2, 1, 2, \dots)$. Its statistical usefulness is justified by the examples given in Section 2 leading to a phase transition. Eq. (S12) becomes

$$ \delta_0^{\bullet} z + \sum_{m \ge 0 \atop 2m+1 \le p} \delta_{2m+1}^{\bullet} \sum_{k_1+\cdots+k_{2m+1} \le p} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_{2m+1}}^{\bullet} z^{k_1+\cdots+k_{2m+1}} \\ + \sum_{m \ge 1 \atop 2m \le p} \delta_{2m}^{\bullet} \sum_{k_1+\cdots+k_{2m} < p} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_{2m}}^{\bullet} z^{k_1+\cdots+k_{2m}+1} = 0. $$

And it can be rewritten using the scrambling operator, which in this case is

$$ \mathfrak{s}_{\beta^{\sim}} \left( \sum_{k=0}^{+\infty} \delta_k^{\bullet} z^k \right) = \frac{1}{z^2} \left( \sum_{k=0}^{+\infty} \delta_{2k+1}^{\bullet} z^{2k+1} + z \sum_{k=0}^{+\infty} \delta_{2k}^{\bullet} z^{2k} \right). $$

**Lemma 6.** Assume that $\delta_1 = -1$. Then $\alpha$ is $(\beta^{\sim}, p)$-related to $\delta$ if and only if $\alpha_1^{\bullet} = \delta_0^{\bullet}, \alpha_{2q}^{\bullet} = 0$ for $2q \le p$ and

$$ \alpha_{2q+1}^{\bullet} = \sum_{m=2 \atop m \text{ even}}^{2q} \delta_m^{\bullet} \sum_{k_1+\cdots+k_m=2q} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} + \sum_{m=3 \atop m \text{ odd}}^{2q+1} \delta_m^{\bullet} \sum_{k_1+\cdots+k_m=2q+1} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} $$

for $2q+1 \le p$. In particular, there exists one and only one sequence $\alpha$ which is $(\beta^{\sim}, p)$-related to $\delta$.

Lemma 5 and Lemma 6 prove (S6).

Remark 6. If $\alpha$ is $(\beta^{\sim}, \infty)$-related to $\delta$, then $\{\alpha_k\}_{k=0}^p$ is $(\beta^{\sim}, p)$-related to $\{\delta_k\}_{k=0}^p$.

**Lemma 7.** We consider a sequence $\alpha$ which is $(\beta^{\sim}, +\infty)$-related to $\delta$. Let $p \in \{1, 2, \dots, \infty\}$. We set

$$ f_{\text{odd}}(x) := \sum_{k \ge 0 \atop 2k+1 \le p} \delta_{2k+1} x^{2k+1} \quad \text{and} \quad f_{\text{even}}(x) := \sum_{k \ge 1 \atop 2m \le p} \delta_{2k} x^{2k} $$

and $g(x) := P_p(x; \alpha)$. Assume in addition that $f_{\text{odd}}, f_{\text{even}} \in \mathcal{A}_{r_f}$ and $g \in \mathcal{A}_{r_g}$ with $g(B_{r_g}) \subset B_{r_f}$ (necessary if $p = \infty$). Then

$$ \mathcal{T}_p (f_{\text{odd}}(g(x)) + x f_{\text{even}}(g(x))) = -\delta_0 x \quad \text{for } x \in B_{r_f}. \qquad (S15) $$

Besides, if $\delta_0\delta_1 \ne 0$, then there exists an analytic function $h$ that solves (S15) (with $h$ instead of $g$) and therefore $\alpha_k := D^k h(0)/k!$ for any $k \ge 1$.
*Proof.* We restrict to the case $p = \infty$, see Remark 6. Let us introduce a new parameter $\nu$ and we define

$$ \tilde{f}(x, \nu) := f_{\text{odd}}(x) + \nu f_{\text{even}}(x) $$

and the analogous $\tilde{f}^{\bullet}(z, \nu)$ with $\delta^{\bullet}$ and $g^{\bullet}(z) := P_{\infty}(z; \alpha^{\bullet})$. As in Lemma 2, if $\alpha^{\bullet}, \gamma^{\bullet} \in \mathbb{R}(\delta^{\bullet})$ and $\delta^{\bullet}$ are linked by the relation

$$ \begin{aligned} & \sum_{m \ge 0} \delta_{2m+1}^{\bullet} \sum_{k_1, \dots, k_{2m+1} \ge 0} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_{2m+1}}^{\bullet} z^{k_1 + \cdots + k_{2m+1}} \\ & \qquad + \nu^{\bullet} \sum_{m \ge 1} \delta_{2m}^{\bullet} \sum_{k_1, \dots, k_{2m} \ge 0} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_{2m}}^{\bullet} z^{k_1 + \cdots + k_{2m}} \\ & = \sum_{m \ge 0} \gamma_{2m+1}^{\bullet} z^{2m+1} + \nu^{\bullet} \sum_{m \ge 1} \gamma_{2m}^{\bullet} z^{2m} \quad (\text{S16}) \end{aligned} $$

in $\mathbb{R}(\delta^{\bullet})[[z, \nu^{\bullet}]]$ for some indeterminate $\nu^{\bullet}$, then

$$ \tilde{f}^{\bullet}(g^{\bullet}(x), \nu^{\bullet}) = \sum_{m \ge 0} \gamma_{2m+1}^{\bullet} x^{2m+1} + \nu^{\bullet} \sum_{m \ge 1} \gamma_{2m}^{\bullet} x^{2m}. $$

If $\alpha$ is $(\beta^{\sim}, +\infty)$-related to $\delta$, then (S16) holds for $\nu^{\bullet} = z$, $\gamma_k = 0$ for $k = 0, 2, 3, \dots$ and $\gamma_1 = -\delta_0$ with $\gamma_k := e(\gamma^{\bullet})$. This proves (S15).

We set $q(y, x) = \delta_0 x + \tilde{f}(y, x)$. If $\delta_1 \ne 0$, then $D_y q(y, 0) \ne 0$. From the analytic implicit function theorem (see e.g., [1]), there exists a unique analytic function $h$ defined in a neighborhood of 0 such that $q(h(x), x) = 0$, meaning that $h$ solves $f_{\text{odd}}(h(x)) + x f_{\text{even}}(h(x)) = -\delta_0 x$ and so (S15). By uniqueness in a neighborhood of 0, $h = g$. $\square$

## S2.6 The case “up-flat”

We consider the case $\beta^{\sim} = (1, 2, \dots, 2)$, which we call the case “up-flat”. With Remark 4, (S12) in the definition of $(\beta^{\sim}, p)$-related sequences (Definition 4) can be replaced by

$$ \delta_0^{\bullet} z + \sum_{m=1}^{p} \delta_m^{\bullet} \sum_{k_1 + \cdots + k_m \le p} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} z^{k_1 + \cdots + k_m} = 0. \quad (\text{S17}) $$

We now give an explicit, recursively computable expressions for the $\alpha_k$'s from the $\delta_k$. Other ways to express the $\alpha_k$'s are possible, for example using Bell polynomials. See also [4, 10] for considerations on the efficiency of numerical implementations.
**Lemma 8.** Assume that $\delta_1 \neq 0$. Then $\alpha$ is $(\beta^-, p)$-related to $\delta$ if and only if

$$ \alpha_1^{\bullet} = \frac{-\delta_0^{\bullet}}{\delta_1^{\bullet}} \quad \text{and} \quad \alpha_q^{\bullet} = - \sum_{m=2}^{q} \frac{\delta_m^{\bullet}}{\delta_1^{\bullet}} \sum_{k_1+\cdots+k_m=q} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} \quad \text{for } 2 \le q \le p. $$

In particular, there exists one and only one sequence $\alpha$ which is $(\beta^-, p)$-related to $\delta$. Besides, each of the $\alpha_q^{\bullet}$ is a polynomial expression in $\delta_m^{\bullet}$, $m = 0, \dots, q$ whose term in $\delta_0^{\bullet}$ has degree $q$.

*Proof.* We rewrite (S17) as

$$ \delta_0^{\bullet} z + \sum_{q=1}^{p} \sum_{m=1}^{p} \delta_m^{\bullet} \left( \sum_{k_1+\cdots+k_m=q} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} \right) z^q = 0. $$

We identify $\alpha_k^{\bullet}$'s according to the power of $z$. In particular, $\delta_0^{\bullet} + \delta_1^{\bullet} \alpha_1^{\bullet} = 0$ so that $\alpha_1^{\bullet} = -\delta_0^{\bullet}/\delta_1^{\bullet}$. Since $\alpha_0^{\bullet} = 0$, for $2 \le q \le p$,

$$ 0 = \sum_{m=1}^{p} \delta_m^{\bullet} \sum_{k_1+\cdots+k_m=q} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet} = \delta_1^{\bullet} \alpha_q^{\bullet} + \sum_{m=2}^{q} \delta_m^{\bullet} \sum_{\substack{1 \le k_1, \dots, k_m < q \\ k_1+\cdots+k_m=q}} \alpha_{k_1}^{\bullet} \cdots \alpha_{k_m}^{\bullet}. $$

This proves that a $(\beta^-, p)$-related sequence may be constructed for any $\delta$ with $\delta_1 \neq 0$. The converse is proved by the same argument of identifying the coefficients in the power of $z$.

The form of the expression and the degree of the term $\delta_0^{\bullet}$ is proved by a direct induction. $\square$

Lemma 5 and Lemma 8 prove (S5).

We now consider an alternative way to compute the coefficients in terms of the inverse of an analytic function. Let us consider the scrambling operator and (S13), which was an alternative expression to (S12), in this case becomes the following alternative expression to (S17):

$$ 0 = s_{\beta^-} z_2 \circ \mathfrak{p}_p \circ P_p(\cdot; \delta^{\bullet}) \circ P_p(z; \alpha^{\bullet}) = \mathfrak{p}_p \circ P_p(\cdot; \delta^{\bullet}) \circ P_p(z; \alpha^{\bullet}) - \delta_0^{\bullet} + \delta_0^{\bullet} z. $$

Therefore, (S12) in Definition 4 (represented here by (S17)) can be replaced by

$$ \mathfrak{p}_p \circ f^{\bullet} \circ P_p(z; \alpha^{\bullet}) = -\delta_0^{\bullet} z \qquad (S18) $$

$$ \text{with } f^{\bullet}(z) := P_p(z; \delta^{\bullet}) - \delta_0^{\bullet} = \sum_{k=1}^{p} \delta_k^{\bullet} z^k. $$
**Remark 7.** To prove the equivalence of (S17) and (S18) we could also use Lemma 2, if $p \in \mathbb{N}$. Indeed (S17) is equivalent to (S9) in Lemma 2 with $\eta_0 = \delta_0$, $\eta_1 = -\delta_0$ and $\eta_k = 0$ for $k \ge 2$ (let us recall that $\alpha_0 = 0$ by definition).

We then see that to get $\alpha$, we compute the compositional inverse of a power series. Hence, our problem is related to the Lagrange inversion formula [6]. Another consequence is that one may look at other ways than series expansions to compute approximations of the root, for example using continued fractions or close form expressions.

The next lemma, Lemma 9, ensures that identifying the inverse of a function constructed as a series/p polynomial of coefficients $\delta_k$'s yields the coefficients $\alpha_k$'s.

**Lemma 9.** Let $p \in \{1, 2, \dots, \infty\}$, $\alpha$ be $(\beta^{-}, p)$-related to $\delta$, and $f(x) := \sum_{k=1}^{p} \delta_k x^k = P_p(x; \delta) - \delta_0$ and $g(x) := P_p(x; \alpha)$. If $p = \infty$, assume in addition Hypothesis 10. Recall that $g(0) = 0$. Then

$$ \mathcal{T}_p(f \circ g(x)) = \mathfrak{p}_p(f \circ g(x)) = -\delta_0 x. $$

If in addition $\delta_0 \neq 0$ and $\delta_1 \neq 0$ then $f$ is invertible on an open neighborhood of 0 and

$$ \alpha_k = (-\delta_0)^k D^k f^{-1}(0)/k! \quad \text{for any } k \ge 1. $$

*Proof.* The first part is a direct consequence of the combination of dummization-evaluation, Eq. (S18) (see Lemmas 3-4). The second part of the statement follows from the Inverse Function Theorem for analytic functions [1] which one may applies when $\delta_1 = Df(0) \neq 0$. The proof is completed, by uniqueness shown in Lemma 8. $\square$

The next corollary follows from the fact that $\mathcal{T}_p(f \circ g) = \mathfrak{p}_p(\mathcal{T}_p f \circ \mathcal{T}_p g)$.

**Corollary 1.** Let $p \in \{1, 2, \dots, \infty\}$, $\alpha$ be $(\beta^{-}, p)$-related to $\delta$, and let $f, g$ functions of class $\mathcal{C}^p$ (if $p = \infty$, assume in addition Hypothesis 10) such that $\mathcal{T}_p f(x) = \sum_{k=1}^{p} \delta_k x^k = P_p(x; \delta) - \delta_0$ and $\mathcal{T}_p g(x) = P_p(x; \alpha)$. Then

$$ \mathcal{T}_p(f \circ g(x)) = -\delta_0 x. $$

**Remark 8.** Let $p \neq \infty$. If $\alpha$ is $(\beta^{-}, \infty)$-related to $\delta$, then $\{\alpha_k\}_{k=0}^p$ is $(\beta^{-}, p)$-related to $\{\delta_k\}_{k=0}^p$.

The above remark ensures that we can take always $p = +\infty$. For instance taking $\delta_k = 0$ for $k$ above a given threshold, say $p$, and we associate with $\delta$ an infinite sequence $\alpha$.
**Example 3.** Let $\delta_k = (-1)^k/\theta^{k-1}$ (resp. $\delta_k = 1/\theta^{k-1}$) for $k \ge 1$ and some $\theta \in \mathbb{R}$.
Then for $|x| < |\theta|$,

$$f(x) := \sum_{k=1}^{\infty} \delta_k x^k = -\frac{\theta x}{\theta + x} \quad (\text{resp. } f(x) = \frac{\theta x}{\theta - x}).$$

Since $f^{-1}(x) = f(x)$ (resp. $f^{-1}(x) = -f(-x) = \frac{\theta x}{\theta+x}$) then $\alpha_k = \frac{\delta_0^k}{\theta^{k-1}}$ (resp. $\alpha_k = -\frac{\delta_0^k}{\theta^{k-1}}$) for $k \ge 1$.

**Example 4.** Let $\psi$ be an analytic function around 0 with $\psi'(0) \ne 0$ (so that $\psi$ is invertible around 0) and set

$$\delta_k := -\frac{1}{k!} \frac{D^k \psi(0)}{\psi'(0)} \quad \text{for } k \ge 1$$

as well as

$$f(z) := \sum_{k \ge 1} \delta_k z^k = \sum_{k \ge 1} \frac{-1}{k!} \frac{D^k \psi(0)}{\psi'(0)} z^k = \frac{\psi(0) - \psi(z)}{\psi'(0)}.$$

Then $f^{-1}(z) = \psi^{-1}(\psi(0) - z\psi'(0))$. By Faà di Bruno formula (S10) $D^k f^{-1}(z) = D^k \psi^{-1}(\psi(0) - z\psi'(0))\psi'(0)^k$ and

$$\alpha_k = \frac{(-\psi'(0)\delta_0)^k D^k \psi^{-1}(\psi(0))}{k!} = \frac{\psi(0)^k D^k \psi^{-1}(\psi(0))}{k!}.$$

We now give a non trivial example.

**Example 5.** Consider $\theta \notin \{0, 1\}$. Let us assume that $\delta_1 = -1$ and $\delta_k = (-\theta)^k$, $k \ge 2$. Then for $|z| < 1/|\theta|$,

$$f(z) = -z + \sum_{k \ge 2} (-\theta)^k z^k = \frac{1}{1 + \theta z} - 1 + (\theta - 1)z.$$

To compute the inverse $f^{-1}$, we are looking for the solution $h(z)$ to the quadratic equation

$$\frac{1}{1 + \theta h(z)} + (\theta - 1)h(z) = 1 - z, \quad \text{for any } z \text{ small enough.}$$

We assume $\theta \in (0, 1)$. The possible solutions are

$$h_{\pm}(z) = \frac{1 - \theta z \pm \sqrt{\theta^2 z^2 - 4\theta^2 z + 2\theta z + 1}}{2\theta(\theta - 1)},$$

where the quantity in the square root is always positive. But $f(0) = 0$ hence $f^{-1} = h_-$.
One could then check that $h_-$ has the same series expansion in $z$ as the one in Lemma 8. It is also possible to express $h_-$ using a continued fraction.
**Corollary 2.** Let $r > 0$ and let $\{f_n\}_n$ be a sequence of functions in $\mathcal{A}_r$ and $f \in \mathcal{A}_r$ be such that $f_n$ converge uniformly to $f$. Then $\delta_k^n := D^k f_n(0)/k!$ converges to $\delta_k$ for any $k \ge 1$. Besides, if $\delta_1 \ne 0$, then for $n$ large enough, the corresponding sequence $\alpha^n$ which is $(\beta^-, +\infty)$-related to $\delta^n = \{\delta_k^n\}_{k \ge 0}$ converges to $\alpha$, the sequence which is $(\beta^-, +\infty)$-related to $\delta = \{\delta_k\}_{k \ge 0}$.

## S2.7 $(\beta, p)$-related sequences in the context of Theorem 4

We have already seen in Lemma 5 how to derive a sequence $\alpha(s)$ satisfying (S3): by finding a $(\beta, p)$-related sequence to $\delta^F(s)$. Hence, when $\beta = \beta^-$ (resp. $\beta = \beta^+$), Lemma 8 (resp. Lemma 6) provides a recursive expression of the desired sequence that we have given in (S5) (resp. (S6)). We now apply Lemma 9 to provide a different expression in the case $\beta = \beta^-$ in this context.

**Proposition 5.** Let $F$ be a function of class $\mathcal{C}^{p+1}(\Theta, \mathbb{R})^S$. Take $\theta_0 \in \Theta$ and consider Notation 10. Assume that Hypotheses (ii)-(iv) of Theorem 4 holds, so that in particular, $F(s, \cdot)$ is invertible around $\theta_0$. Define

$$ \mathcal{C}_{\theta_0} F(s, x) := F^{-1} \left( s, F(s, \theta_0) \left( 1 - \frac{x}{\varphi(s)} \right) \right) - \theta_0. $$

Let $\alpha_0 := 0$ and $\alpha_k$ be the $k$-th Taylor coefficient of $\mathcal{C}_{\theta_0} F(s, \cdot)$ for $k \ge 1$, that is

$$ \alpha_k(s) := \frac{D^k \mathcal{C}_{\theta_0} F(s, 0)}{k!} = \frac{D^k F^{-1}(s, F(s, \theta_0))}{k!} \left( - \frac{\{F\}_0(s)}{\{F\}_1(s)} DF(s, \theta_0) \right)^k. $$

Then $\{\alpha_k(s)\}_{k=0}^p$ is $(\beta^-, p)$-related to $\delta^F(s)$, it satisfies equation (S3) (i.e. Item (i) of Theorem 4), and

$$ \begin{aligned} \theta(s) &= \theta_0 + \mathcal{C}_{\theta_0} F(s, \varphi(s)) \\ &= \theta_0 + \mathcal{T}_p \mathcal{C}_{\theta_0} F(s, \varphi(s)) + O(\varphi(s)^{p+1}) = \theta_0 + \sum_{k=1}^p \alpha_k(s) \varphi(s)^k + O(\varphi(s)^{p+1}). \end{aligned} $$

*Proof.* By construction, for any $x$ small enough,

$$ \frac{F(s, \theta_0 + \mathcal{C}_{\theta_0} F(s, x)) - F(s, \theta_0)}{DF(s, \theta_0)} = \frac{-F(s, \theta_0)}{DF(s, \theta_0)} \frac{x}{\varphi(s)}. \quad (S19) $$

In particular, $\mathcal{C}_{\theta_0} F(s, 0) = 0$ and $F(s, \theta_0 + \mathcal{C}_{\theta_0} F(s, \varphi(s))) = 0$ so that the root $\theta(s)$ to $F(s, \theta) = 0$ is $\theta(s) = \theta_0 + \mathcal{C}_{\theta_0} F(s, \varphi(s))$.
We set {$\delta_k$}$_k := \delta$. Then, inspired by Example 4, we set

$$ \psi(s, z) = \frac{F(s, \theta_0) - F(s, \theta_0 + z)}{DF(s, \theta_0)}. $$

Hence $\mathcal{T}_p\psi(s, z) = \sum_{k=1}^p \delta_k(s)z^k$ which is analytic. From (S19),

$$ \psi(s, \mathcal{E}_{\theta_0} F(s, x)) = -\delta_0(s)x. $$

Note that

$$ \mathfrak{p}_p(\mathcal{T}_p\psi(s, \mathcal{T}_p\mathcal{E}_{\theta_0}F(s, x))) = \psi(s, \mathcal{E}_{\theta_0}F(s, x)) = -\delta_0(s)x. $$

This ensures by Eq. (S18), together with Lemmas 3-4, that the coefficients $\alpha(s)$ of the Taylor expansion of $\mathcal{E}_{\theta_0}F(s, \cdot)$ around 0 are $(\beta^{-}, p)$-related to $\delta^F(s)$. Hypothesis (i) is satisfied, so that the conclusion of Theorem 4 holds. $\square$

## S2.8 Perturbation of the roots and $(\beta, p)$-related sequences

Let $\beta \in \{\beta^{-}, \beta^{\sim}\}$. Let $\Lambda: \mathbf{S} \to \mathbb{R}$ and set $\{\Lambda\}_0(s) := \varphi(s)^{\beta_0}\Lambda(s)$. Let us consider solving

$$ F(s, \theta_{\Lambda}(s)) = \Lambda(s). $$

We define

$$ \{F_{\Lambda}\}_k(s) := \begin{cases} \{F\}_0(s) - \{\Lambda\}_0(s) & \text{for } k = 0, \\ \{F\}_k(s) & \text{for } k > 0 \end{cases} $$

as well as the corresponding $\delta_{\Lambda k}(s)$ and $\alpha_{\Lambda k}(s)$.

**Proposition 6.** For each $k \ge 1$, the coefficients $\alpha_{\Lambda k}(s)$ are polynomial expressions in $\{F\}_\ell(s)/\{F\}_1(s)$, $\ell \le k$ and $\{\Lambda\}_0(s)/\{F\}_1(s)$. Moreover, if $\{\Lambda\}_0/\{F\}_1$ converges to 0 with $s$ and provided that $\alpha_k$ converges, then $\alpha_{\Lambda k}$ and $\alpha_k$ have the same limit for each $k \ge 0$.

*Proof.* The proof is immediate once it has been noticed by $\delta_{\Lambda k} = \delta_k$ for $k > 1$ and $\delta_{\Lambda 0} = \delta_0 - \{\Lambda\}_0$. The expression of $\alpha_{\Lambda k}(s)$ in terms of $\{F\}_\ell(s)/\{F\}_1(s)$ and $\Lambda(s)/\{F\}_1(s)$ are obtained by the Newton formula. $\square$
# S3 Reparametrization and Changes of scale

## S3.1 Reparametrization through changes of variables

In the context of statistical estimation, it is sometimes convenient to perform some change of variable on the space of parameters.

Let us consider a one-to-one map $\Phi$ from the space of parameters $\Theta$ to the space $\Theta^\dagger$ with inverse $\Psi$. We assume that $\Phi \in \mathcal{C}^{p+1}$. For the sake of simplicity, we assume that 0 belongs to the interiors of both $\Theta$ and $\Theta^\dagger$. Moreover, we assume that $\Phi(0) = 0$.

**Notation 14 (Pullback).** For a function $f: \Theta \to \mathbb{R}$, we set $\Psi^*f := f \circ \Psi$ so that $\Psi^*f$ is a function from $\Theta^\dagger$ to $\mathbb{R}$. The operator $\Psi^*$ is the pullback. For $f: \mathbf{S} \times \Theta \to \mathbb{R}$, we extend the pullback to $\Psi^*f(s, \eta) = f(s, \Psi(\eta))$ for $(s, \theta) \in \mathbf{S} \times \Theta^\dagger$. For a function $F \in \mathcal{C}^{p+1}(\Theta, \mathbb{R})^\mathbf{S}$, we define $F^\dagger(s, \eta) := \Psi^*F(s, \eta) = F(s, \Psi(\eta))$ which is a function in $\mathcal{C}^{p+1}(\Theta^\dagger, \mathbb{R})^\mathbf{S}$. We also set $\{F^\dagger\}_k(s) := \varphi(s)^{\beta_k} D^k F^\dagger(s, 0)$ for $k = 0, \dots, p$.

We focus on the case “up-flat” ($\beta = \beta^-$). From the Faà di Bruno formula (Proposition 4), we see that $\{\{F^\dagger\}_k(s)\}_{k=0,\dots,p}$ converges whenever $\{\{F\}_k(s)\}_{k=0,\dots,p}$ does. This is not necessarily true in the case “zig-zag”. Indeed, let $\varphi(s) \in (-1, 1)$, then in the “zig-zag” case, $\varphi(s)^{-1}\{F\}_1(s)$ may diverge and so

$$ \{F^\dagger\}_2(s) = \{F\}_2(s)(D\Psi(0))^2 + \varphi(s)^{-1}\{F\}_1(s)D^2\Psi(0) $$

does not converge neither. The same happens for $\{F^\dagger\}_{2k}(s)$, by Faà di Bruno formula (S10) (in Proposition 4). So, starting from “zig-zag” case, taking $F^\dagger$ we bump into the “up-flat” case. More generally, if $\varphi(s) \in (-1, 1)$, starting from $\beta = (\beta_0, \beta_1, \beta_2, \dots, \beta_p)$, we bump into $(\beta_0, \beta_1, \beta_2^*, \beta_3^*, \dots, \beta_p^*)$ where $\beta_k^* := \max_{n \in \{1, \dots, k\}} \beta_n$. This latter kind of coefficient, as $\beta^-$, is “stable” under such changes of variables.

**Proposition 7.** Fix $s \in \mathbf{S}$. Let us define $\delta = \{\delta_k\}_{k=0,\dots,p}$ and $\delta^\dagger = \{\delta_k\}_{k=0,\dots,p}$ by

$$ \delta_k(s) := \frac{\{F\}_k(s)}{k!} \quad \text{and} \quad \delta_k^\dagger(s) := \frac{\{F^\dagger\}_k(s)}{k!} \quad \text{for } k = 0, 1, \dots, p. $$

Let $\alpha(s)$ (resp. $\alpha^\dagger(s)$) be the unique $(\beta^-, p)$-related sequence to $\delta(s)$ (resp. $\delta^\dagger(s)$) given in Lemma 8. Then

$$ \alpha_q(s) = \sum_{m=1}^{q} \frac{D^m \Psi(0)}{m!} \sum_{k_1+\dots+k_m=q} \alpha_{k_1}^{\dagger}(s) \dots \alpha_{k_m}^{\dagger}(s) \text{ for } q=1,\dots,p, $$
$$ (F, \theta) \xrightarrow{\text{expansion}} \sum_{k=1}^{p} \alpha_k \cdot z^k \xrightarrow{\text{evaluation}} \theta_0 + \sum_{k=1}^{p} \alpha_k \varphi^k \\ \Big| \Psi^* \times \Phi \qquad \mathfrak{p}_p \circ \mathcal{T}_p^\bullet \Psi \Big| \\ (F^\dagger, \theta^\dagger) \xrightarrow{\text{expansion}} \sum_{k=1}^{p} \alpha_k^\dagger \cdot z^k \xrightarrow{\text{evaluation}} \theta_0^\dagger + \sum_{k=1}^{p} \alpha_k^\dagger \varphi^k $$

<caption>Figure S1: Change of variable.</caption>

or, in other words,

$$ \sum_{k=1}^{p} \alpha_k(s) \cdot z^k := \mathfrak{p}_p \circ \mathcal{T}_p^\bullet \Psi \left( \sum_{k=1}^{p} \alpha_k^\dagger(s) \cdot z^k \right), $$

where $\mathcal{T}_p^\bullet \Psi \in \mathbb{R}(\delta^\bullet(s))[[z]]$ is the formal Taylor expansion of $\Psi$ (see Definition 3).

The meaning of this proposition is that expanding the root $\theta^\dagger(s)$ of $F^\dagger(s, \theta^\dagger(s)) = 0$ in terms of $\varphi(s)$ using our procedure of Lemma 8 and then extending $\Psi(\theta^\dagger(s))$ using a Taylor series leads to the same expansion as the one given computing the root $\theta(s)$ of $F(s, \theta(s)) = 0$ using Lemma 8. This is summarized in the diagram of Figure S1.

*Proof.* In this proof let $F_\bullet := \mathcal{T}_p^\bullet F$ and $F_\bullet^\dagger := \mathcal{T}_p^\bullet F^\dagger$ from Definition 3. By (S14) and Lemma 8, the sequence $\alpha(s) \cdot \in \mathbb{R}(\delta^\bullet)^p$ corresponding to $\alpha(s)$ is the unique solution to

$$ \mathfrak{s}_\beta \dashv \mathfrak{p}_p \circ F_\bullet(s, P(z)) = 0 \text{ with } P(z) := \sum_{k=1}^{p} \alpha_k(s) \cdot z^k \text{ and } \alpha_k(s) \cdot \in \mathbb{R}(\delta(s)^\bullet). \quad (S20) $$

Similarly, $\alpha^\dagger(s) \cdot$ is the unique solution to

$$ \mathfrak{s}_\beta \dashv \mathfrak{p}_p \circ F_\bullet^\dagger(s, P^\dagger(z)) = 0 \text{ with } P^\dagger(z) := \sum_{k=1}^{p} \alpha_k^\dagger(s) \cdot z^k = 0 $$

and $F_\bullet^\dagger(z) = \sum_{k=0}^{p} \delta_k^\dagger(s) z^k \in \mathbb{R}[\delta^\dagger(s)^\bullet][[z]].$

The $\alpha_k^\dagger(s) \cdot$'s are rational functions in $\mathbb{R}(\delta^\dagger(s)^\bullet)$. From the very definition, $F^\dagger(s, \eta) := F(s, \Psi(\eta))$. By the Faà di Bruno formula (Proposition 4), $\{F^\dagger\}_k(s)$ is a linear combination of the elements $\{F\}_\ell(s)$, $\ell = 1, \dots, k$ and the derivatives of $\Psi$. Therefore, the $\alpha_k^\dagger(s) \cdot$'s and $\delta_k^\dagger(s)^\bullet$'s can be seen as rational functions in $\mathbb{R}(\delta(s)^\bullet)$. Thus $F_\bullet^\dagger(z)$ and $\mathcal{T}_p^\bullet \Psi \circ P^\dagger(z)$ can be seen as elements of $\mathbb{R}[\delta(s)^\bullet][[z]].$
By Lemma 3,

$$ \mathfrak{p}_p \circ F_{\bullet}^{\dagger} = F_{\bullet}^{\dagger} = \mathcal{T}_p^{\bullet}(\Psi^*F) = \mathfrak{p}_p \circ (\mathcal{T}_p^{\bullet}\Psi)^*F_{\bullet}. $$

Since $\mathcal{T}_p^{\bullet}\Psi \circ P^{\dagger}(z) \in \mathbb{R}[\delta(s)^{\bullet}][[z]]$ and

$$ 0 = s_{\beta} \negthinspace \circ \negthinspace \mathfrak{p}_p \circ F_{\bullet}^{\dagger}(s, P^{\dagger}(z)) = s_{\beta} \negthinspace \circ \negthinspace \mathfrak{p}_p \circ F_{\bullet}(s, \mathcal{T}_p^{\bullet}\Psi \circ P^{\dagger}(z)). $$

Hence $\mathcal{T}_p^{\bullet}\Psi \circ P^{\dagger}(z)$ solves (S20), which is known to have a unique solution, $P(z)$. Thus $P(z) = \mathcal{T}_p^{\bullet}\Psi \circ P^{\dagger}(z)$. The result follows by an application of the evaluation map. $\square$

## S3.2 Change of scale

We now introduce another change of variable, called *change of scale*, leading to a different expansion. The impact of this change of scale technique leads for instance to obtain a “natural” expansion in the context of exponential families.

Let us take a one-to-one function $\Phi \in \mathcal{C}^{p+1}(\Theta, \Theta^\dagger)$ whose derivative $\Phi'$ never vanishes. Such a function $\Phi$ is seen as a *scale*.

**Notation 15.** For a differentiable function $f: \mathbb{R} \to \mathbb{R}$, we write $D_{\Phi}f = Df/D\Phi$, where D is the derivative operator.

Since we solve $F(s, \theta) = 0$, a natural operation consists in setting

$$ \tilde{F}(s, \theta) := \Phi^\sharp F(s, \theta) := \frac{F(s, \theta)}{D\Phi(\theta)} $$

and to solve $\tilde{F}(s, \theta) = 0$. The root $\theta(s)$ is the same for $F$ and $\tilde{F}$. However, as discussed below, the expansion could be different.

In the case of the MLE, as $F(s, \cdot)$ takes scalar values, there exists a function $G \in \mathcal{C}^{p+2}(\Theta, \mathbb{R})^\mathbf{S}$ such that $DG(s, \cdot) = F(s, \cdot)$. Considering $\tilde{F}(s, \theta)$ corresponds to a change of variable at the level of the function $G$. Indeed, with Notation 15,

$$ D_{\Phi}G(s, \theta) = \frac{DG(s, \theta)}{D\Phi(\theta)} = \tilde{F}(s, \theta) \text{ in } \mathcal{C}^{p+1}(\Theta, \mathbb{R})^{\mathbf{S}}. $$

We set $G^\dagger(s, x) := G(s, \Psi(x))$ and $F^\dagger(s, x) := F(s, \Psi(x))$ so that from standard computations,

$$ D(G^{\dagger}) = (D_{\Phi}G)^{\dagger} \text{ and } F^{\dagger} = D_{\Psi}(G^{\dagger}). $$

Therefore,

$$ \tilde{F}^{\dagger} := (\tilde{F})^{\dagger} = D(G^{\dagger}) \text{ so that } \tilde{F} = \Phi^{\star}D(\Psi^{\star}G). $$
<figure><img src="image_7.png" /><figcaption>Figure S2: Change of scales and change of variable.</figcaption></figure>

We denote by $\tilde{\alpha}(s)$ and $\tilde{\alpha}^{\dagger}(s)$ the unique $(\beta^{-}, p)$-related sequences to $\{\{\tilde{F}\}_k(s)/k!\}_{k=0,...,p}$ and $\{\{\tilde{F}^{\dagger}\}_k(s)/k!\}_{k=0,...,p}$ defined by Lemma 8.

Proposition 7 may be applied to compute $\tilde{\alpha}^{\dagger}(s)$ and then $\tilde{\alpha}(s)$, instead of computing the derivative of $\tilde{F}$. We summarize the various relationships in Figure S2.

Are $P_{\tilde{\alpha}}$ and $P_{\alpha}$ the same?

Let us reduce to the case $\Phi(0) = 0$. If $\Phi(x) = \lambda x$ for some $\lambda \in \mathbb{R}\setminus\{0\}$, then the two expansions coincide by the fact that $F = DG = \lambda D_{\Phi}G$ and Remark 5.

Note that if $\alpha_1 \neq 0$ and $\tilde{\alpha}_1 \neq 0$ then they are different unless $\Phi''(0) = 0$ since

$$ \tilde{\alpha}_1(s) = \frac{\alpha_1(s)}{1 + \frac{\Phi''(0)}{\Phi'(0)} \alpha_1(s) \varphi(s)^{\beta_1 - \beta_0}}. $$

Let us observe that $\tilde{\alpha}_1$ and $\alpha_1$ coincide asymptotically as $s \to \infty$, when $\varphi(s)$ vanishes and $\alpha_1(s)$ converges in distribution.

## S4 An application to statistics of diffusions

We show how to use our method for non i.i.d. random variables and another estimator than the MLE.
Let us consider the inference of the parameter $\theta$ in the stochastic differential equation

$$dX_t^\theta = \theta \cdot b(X_t^\theta) dt + \sigma dB_t, \ t \ge 0.$$

for a Brownian motion $B$, a measurable function $b$ regular enough, and $\sigma \in (0, +\infty)$.

For this, we follow the methodology of [3], which is based on the martingale approximations of the log-likelihood.

We fix a time step $\Delta > 0$. We assume we observe a realization $(X_0, \dots, X_n)$ of the vector $(X_0^\theta, X_1^\theta, \dots, X_n^\theta)$ where $X_i^\theta$ is a shorthand for $X_{i\Delta}^\theta$, $i = 0, \dots, n$.

The unknown parameter $\theta$ is estimated by $\theta(n)$ that solves $F(n, \theta(n)) = 0$ with

$$F(n, \theta) := \sum_{i=1}^{n} \frac{b(X_{i-1})}{\sigma^2} (X_i - M(X_{i-1}, \theta)) \quad (S21)$$

where

$$M(x, \theta) := \mathbb{E}[X_{\Delta}^{\theta} | X_{0}^{\theta} = x].$$

The asymptotic of $\theta(n)$ is studied when $n \to \infty$ while $\Delta$ is kept fixed (long time and fixed time step).

By a straightforward computation,

$$D_{\theta}^{k} F(n, \theta) := - \sum_{i=1}^{n} \frac{b(X_{i-1})}{\sigma^2} D_{\theta}^{k} M(X_{i-1}, \theta).$$

**Hypothesis 11.** The process $X^\theta$ is ergodic. Its invariant measure is its renormalized speed measure

$$\mu_{\theta}(dx) = \frac{1}{A(\theta)} \exp \left( 2\theta \int_{0}^{x} \frac{b(y)}{\sigma^2} dy \right) dx,$$

where $A(\theta)$ is a normalizing constant such that $\int_{\mathbb{R}} \mu_{\theta}(dx) = 1$.

We denote by $(t, x, y) \mapsto p_{\theta}(t, x, y)$ the density transition function of $X^\theta$. For $f: \mathbb{R}^2 \to \mathbb{R}$, we write

$$Q^{\theta}(f) := \int_{\mathbb{R}} \int_{\mathbb{R}} f(x, y) p_{\theta}(\Delta, x, y) \mu_{\theta}(dx).$$

The following proposition is a consequence of ergodicity.

**Proposition 8** (See, e.g., [3, Lemma 3.1]). Let $f: \mathbb{R}^2 \to \mathbb{R}$ be a measurable function such that $Q^{\theta}(f^2) < +\infty$. Then

$$\frac{1}{n} \sum_{i=1}^{n} f(X_{i-1}^{\theta}, X_{i}^{\theta}) \underset{n \to \infty}{\longrightarrow} Q^{\theta}(f)$$
in L²($\mathbb{P}_{\theta}$). If $Q^{\theta}(f) = 0$, then

$$ \frac{1}{\sqrt{n}} \sum_{i=1}^{n} f(X_{i-1}^{\theta}, X_{i}^{\theta}) \xrightarrow{n \to \infty} G \text{ with } G \sim \mathcal{N}(0, Q^{\theta}(f^2)) $$

under $\mathbb{P}_{\theta}$.

Hence, for $k \ge 1$, owing to the definition of the invariant measure, under $\mathbb{P}_{\theta}$, replacing the observations $X_i$ by the corresponding random variable $X_i^{\theta}$,

$$ \frac{-\mathrm{D}_{\theta}^{k} F(n, \theta)}{n} = \frac{1}{n} \sum_{i=1}^{n} \frac{b(X_{i-1}^{\theta})}{\sigma^2} \mathrm{D}_{\theta}^{k} M(X_{i-1}^{\theta}, \theta) \xrightarrow{n \to \infty} \int_{\mathbb{R}} \frac{b(x)}{\sigma^2} \mathrm{D}_{\theta}^{k} M(x, \theta) \mu_{\theta}(dx), $$

while, if $\int_{\mathbb{R}} \frac{b(x)}{\sigma^2} \mathrm{D}_{\theta}^{k} M(x, \theta) \mu_{\theta}(dx) = 0$,

$$ \frac{1}{\sqrt{n}} \mathrm{D}^k F(n, \theta) = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} \frac{b(X_{i-1}^{\theta})}{\sigma^2} \mathrm{D}_{\theta}^{k} M(X_{i-1}^{\theta}, \theta) \xrightarrow{n \to \infty} \mathcal{N}(0, c^2) $$

with

$$ c^2 = \int_{\mathbb{R}} \int_{\mathbb{R}} p(\Delta, x, y) \frac{b(x)^2}{\sigma^4} (y - M(x, \theta))^2 \mu_{\theta}(dx) dy. $$

We note that the quantity $M(x, \theta)$ cannot be always computed in an explicit manner, because

$$ M(x, \theta) = \mathbb{E}_{\theta}[X_{\Delta}^{\theta} | X_{0}^{\theta} = x] = x + \theta \int_{0}^{\Delta} \mathbb{E}_{\theta}[b(X_{s}^{\theta}) | X_{0}^{\theta} = x] ds. \quad (S22) $$

**Example 6 (Ornstein-Uhlenbeck process).** Let us consider that $b(x) = -x$ and $\theta > 0$ so that $X^{\theta}$ is the Ornstein-Uhlenbeck process. From (S22),

$$ M(x, \theta) = x \exp(-\Delta\theta) \text{ and } \mu_{\theta}(dx) = \frac{\sqrt{\theta}}{\sqrt{\pi\sigma^2}} \exp\left(-\frac{\theta x^2}{\sigma^2}\right) dx $$

which is a centered Gaussian law of variance $\sigma^2/(2\theta)$. Therefore, $\mathrm{D}^k M(x, \theta) = x(-\Delta)^k \exp(-\Delta\theta)$. We are then in the case “up-flat”. We consider the change of variable $\eta := \Phi(\theta) := e^{-\Delta\theta}$ (see Section S3.1) and we get

$$ F_{n}^{\dagger}(\eta) = \sum_{i=1}^{n} \frac{-X_{i-1}}{\sigma^2} (X_i - X_{i-1}\eta). $$

It holds that $\mathrm{D}_{\eta} F_{n}^{\dagger} = \sum_{i=1}^{n} \frac{1}{\sigma^2} (X_{i-1})^2$ and $\mathrm{D}_{\eta}^{k} F_{n}^{\dagger} = 0$ for $k \ge 2$. By Theorem 6,

$$ \eta(n) = \eta - \frac{F_{n}^{\dagger}(\eta)}{\mathrm{D}_{\eta} F_{n}^{\dagger}(\eta)} = \frac{\sum_{i=1}^{n} X_{i-1} X_{i}}{\sum_{i=1}^{n} (X_{i-1})^2} $$
so that

$$ \theta(n) = \frac{-1}{\Delta} \log \frac{\sum_{i=1}^{n} \mathbf{X}_{i-1} \mathbf{X}_{i}}{\sum_{i=1}^{n} (\mathbf{X}_{i-1})^{2}} $$

We have the same formula as the one in [3, Example 2.1, p. 21], which is obtained by direct means (note that we use $b(x) = -x$ instead of $b(x) = x$ as in [3]).

**Example 7** (Using an approximation of $M(x, \theta)$). When no close form solution to $M(x, \theta)$ (see (S22)), we consider that we have a smooth approximation $M_{\delta}(x, \theta)$ of $M(x, \theta)$ depending on a parameter $\delta$. We set

$$ \epsilon_{\delta}(\theta) := \sum_{i=1}^{n} \frac{b(\mathbf{X}_{i-1})}{\sigma^2} (M_{\delta}(\mathbf{X}_{i-1}, \theta) - M(\mathbf{X}_{i-1}, \theta)) $$

and define $F_{\delta,n}(\theta)$ similarly to $F(n, \theta)$ in (S21) with $M_{\delta}$ instead of $M$. The solution $\theta_{\delta,n}$ to $F_{\delta,n}(\theta_{\delta,n}) = 0$ also solves

$$ F(n, \theta_{\delta,n}) = \epsilon_{\delta}(\theta_{\delta,n}). $$

Assume that we are in the case “up-flat” and that $\epsilon_{\delta}(\theta_{\delta,n})/\sqrt{n}$ converges to 0 as $(\delta, n)$ converges to $(0, +\infty)$ as a net (this could imply some relationship between the rate of convergence of $\delta$ to 0 and the one of $n$ to $+\infty$). Proposition 2 allows one to conclude that $\theta_{\delta,n}$ and $\theta(n)$ have the same asymptotic behavior.

## References

[1] R. Abraham, J. E. Marsden, and T. Ratiu. *Manifolds, tensor analysis, and applications*. Second. Vol. 75. Applied Mathematical Sciences. Springer-Verlag, New York, 1988. DOI: 10.1007/978-1-4612-1029-0.

[2] G. Allaire, A. Lamacz-Keymling, and J. Rauch. “Crime pays; homogenized wave equations for long times”. In: *Asymptotic Anal.* 128.3 (May 2022), 295–336. DOI: 10.3233/asy-211707.

[3] B. M. Bibby and M. Sørensen. “Martingale Estimation Functions for Discretely Observed Diffusion Processes”. In: *Bernoulli* 1.1/2 (Mar. 1995), p. 17. DOI: 10.2307/3318679.

[4] R. P. Brent and H. T. Kung. “Fast Algorithms for Manipulating Formal Power Series”. In: *J. ACM* 25.4 (Oct. 1978), pp. 581–595. DOI: 10.1145/322092.322099.
[5] A. Frabetti and D. Manchon. “Five interpretations of Faà di Bruno’s formula”. In: *Faà di Bruno Hopf algebras, Dyson-Schwinger equations, and Lie-Butcher series*. Based on a conference hosted by IRMA, Strasbourg, France, June 27–July 1, 2011. Zürich: European Mathematical Society (EMS), 2015, pp. 91–147.

[6] I. M. Gessel. “Lagrange inversion”. In: *J. Combin. Theory Ser. A* 144 (2016), pp. 212–249. DOI: 10.1016/j.jcta.2016.06.018.

[7] W. P. Johnson. “The curious history of Faà di Bruno’s formula”. In: *Am. Math. Mon.* 109.3 (2002), pp. 217–234. DOI: 10.2307/2695352.

[8] S. G. Johnston and J. Prochno. “Faà di Bruno’s formula and inversion of power series”. In: *Adv. Math.* 395 (Feb. 2022), p. 108080. DOI: 10.1016/j.aim.2021.108080.

[9] R. B. Leipnik and C. E. M. Pearce. “The multivariate Faà di Bruno formula and multivariate Taylor expansions with explicit integral remainder term”. In: *ANZIAM J.* 48.3 (Jan. 2007), pp. 327–341. DOI: 10.1017/s1446181100003527.

[10] J. van der Hoeven. “Relax, but don’t be too lazy”. In: *J. Symb. Comput.* 34.6 (2002), pp. 479–542. DOI: 10.1006/jsco.2002.0562.