1 following inequality

$$|\mathcal{E}(\theta) - \mathcal{E}(\theta')| \leq \langle \bar{\mathbf{a}}' |\theta - \theta'|, |\varphi_{\ell}| \rangle \leq |\theta - \theta'|$$

2 is true with a probability tending to 1. Since $\Theta$ is a compact set, it is possible to find a finite partition

3 $\Theta_1, \dots, \Theta_K$ of $\Theta$ such that the diameter of $\Theta_i$ is smaller than an arbitrary $\alpha$ independently of $n$. This

4 ensures that condition (A.2) is also satisfied and the uniform convergence of $Q_{n,L}(\theta)$ to $Q_L^*(\theta)$ can be

5 claimed. Finally, from lemma 5, we see that there exists an isolated global minimum $\theta_L^*$ of $Q_L^*(\theta)$, i.e.

6 such that for every $\epsilon > 0$

$$\inf_{\theta/|\theta-\theta^*| \ge \epsilon} Q_L^*(\theta) > Q_L^*(\theta_L^*)$$

7 As a consequence, $\hat{\theta}_{n,L}$ converges to $\theta_L^*$ in probability. Moreover, lemma 5 asserts that the bias between

8 $\theta_L^*$ and $\theta^*$ is of order $r_L$, which is the (uniform) approximation error for the set of function $(\mathcal{E}(\phi^*, \theta))_{\theta \in \Theta}$.

9 If the envelope function is sufficiently regular (in $H^1$), and we use the sine basis we get that the uniform approximation error is $o(1/L)$.

10 **Lemma 5.** Under condition C1 et C5, there exists $L_0$ such that for all $L > L_0$, $Q_L^*$ has a local minimum

11 $\theta_L^*$ such that for all $\epsilon > 0$, $\inf_{|\theta-\theta_L^*| > \epsilon} Q_L^*(\theta) > 0$. Moreover, for all $\epsilon' > 0$, there exists $L'_0 > L_0$ such that
12 for all $L > L'_0$

$$|\theta_L^* - \theta^*| \le \epsilon'$$

14 In particular, if $\bar{h}_{\theta\theta}$ is in $H^1$ and $(\varphi_\ell)_{\ell \ge 1}$ is the sine basis, then $\theta_L^* - \theta^* = o(1/L)$.

15 **Proof.** The first step is to show that $Q_L^*(\theta)$ has a local isolated minimum $\theta_L^*$ close to $\theta^*$, and that this distance can be made arbitrarily small. We introduce first some useful notations. Since $(\varphi_\ell)_{\ell \ge 1}$ is an orthonormal basis in $L^2([0, 1])$, we have $\mathbf{E}_F(\phi^*, \theta) = \sum_{\ell \ge 1} e_\ell(\phi^*, \theta) \varphi_\ell$ and $\partial_\theta \mathbf{E}_F(\phi^*, \theta) = \sum_{\ell \ge 1} \partial_\theta e_\ell(\phi^*, \theta) \varphi_\ell$ where for $j = 1, \dots, p$,

$$\partial_{\theta_j} e_\ell(\phi^*, \theta) = \langle \partial_{\theta_j} \mathcal{E}(\phi^*, \theta), \varphi_\ell \rangle = \langle f_{\theta_j}(\cdot, \phi^*, \theta), \varphi_\ell \rangle$$

19 because we can differentiate under the integral sum ($f$ is uniformly $L^2$-Lipschitz in $D \times \Theta$) and we

20 have also the Hessian matrix defined element wise as $\partial_{\theta_i \theta_j} \mathbf{E}_F(\phi^*, \theta) = \sum_{\ell \ge 1} \partial_{\theta_i \theta_j} e_\ell(\phi^*, \theta) \varphi_\ell$ with

21 $\partial_{\theta_i \theta_j} e_\ell(\phi^*, \theta) = \langle f_{\theta_i \theta_j}(\cdot, \phi^*, \theta), \varphi_\ell \rangle$, because $f_\theta(t, x, \theta)$ is also uniformly $L^2$-Lipschitz in $D \times \Theta$. First
$$ \left\{ \begin{array}{l} \frac{d}{dt} X_1 = \rho_m D^m(t - \tau) - \gamma_{ms} \\ \frac{d}{dt} X_2 = \rho_s D^s(t - \tau) - \gamma_{sl} + \gamma_{ms} e^{(X_1 - X_2)} \\ \frac{d}{dt} X_3 = \gamma_{sl} e^{(X_2 - X_3)} - \delta_l \end{array} \right. \qquad (6.1) $$

1 is credible for representing the dynamics of the observations. Model (6.1) is written in log-scale (i.e with $X_1 = \log(T_E^m)$, $X_2 = \log(T_E^s)$ and $X_3 = \log(T_E^l)$), and the parameter $\theta = (\rho_m, \rho_s, \delta_l, \gamma_{ms}, \gamma_{sl})^T$ has to be estimated from the data. The function $D$ and the delay are known (estimated from the data).

4 The available data are the variables $T_E^m$, $T_E^s$ and $T_E^l$ for six different subjects and are measured at times $T = [0, 4, 5, 6, 7, 8, 9, 10, 11, 12, 14, 24]$. Following Wu et al., we stabilize the variance by a log transformation, hence we consider directly the variables $X_i$, $i = 1, 2, 3$. We assume that each subject 6 share the same true parameter $\theta^*$ and the same initial conditions: at each time point, we compute the 8 mean of the log-measurement (over the subjects) as pseudo-observations.

9 We estimate $D^m$ with a spline smoother computed with cubic B-Splines and GCV selection for the 10 knots. As in Wu et al, the nonparametric proxy is a regression spline $\hat{X} = (\hat{X}_1, \hat{X}_2, \hat{X}_3)$ defined on 11 [5, 14]; we do not consider earlier times since the influenza specific CD8+ T cells are not produced 12 before. Since we have a small number of observations, the choice of the knots for the cubic splines is 13 done manually.

14 Nevertheless for the parameter estimation, we have tested several estimates $\hat{X}$ (with different knots 15 locations), and different number of tests functions $L$: we selected $L = 3$ or $L = 4$. The corresponding 16 estimators are denoted $\hat{\theta}_3^{OC}$ and $\hat{\theta}_4^{OC}$. Moreover, in order to improve the accuracy, we have used a 17 weighted version of the OC estimator, similar to the classical "Generalized Methods of Moments" (this 18 procedure is detailed in section 5 of Supplementary Material I). The quality of the estimator is evaluated 19 by the SSE:

$$ SSE = \sum_{s=1}^{6} \sum_{d=1}^{3} \sum_{i=1}^{N} (y_{i,d,s} - \phi_d(t_i, \hat{\theta}, \hat{X}(0)))^2 $$

20 where $y_{i,d,s}$ is the observation at time $t_i$ for the $s$-th subject for the transformed variable $X_d$. As suggested 21 in Wu et al, we use the OC estimates as initial values for NLS estimation. For both estimates, we obtain 22 the same estimator which is then simply denoted as $\hat{\theta}^{NLS}$. We provide three different estimates $\hat{\theta}_3^{OC}$, $\hat{\theta}_4^{OC}$ 23 and $\hat{\theta}^{NLS}$; we mention also $\hat{\theta}^{ref}$, which is the estimate obtained in Wu et al [42].
of all, we remark that $\mathcal{E}(\phi^*, \theta)$ is bounded by $\bar{h} \in L^2$ (uniformly in $\theta$, condition C2), which implies that
$\|\mathbf{E}_L(\phi^*, \theta)\|_{L^2}^2$ converges also uniformly to $\|\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)\|_{L^2}^2$. Let $\epsilon' > 0$, since $\theta^*$ is a local strict minimum
of $Q_{\mathcal{F}}$, there exists $L_0$ such that for all $L > L_0$, $Q_L^*$ has a strict local minima such that $|\theta_L^* - \theta^*| \le \epsilon'$.
Moreover, by uniform convergence, we have also that $\inf_{|\theta - \theta_L^*| > \epsilon} Q_L^*(\theta) > 0$.

Now, we relate $\theta_L^* - \theta^*$ to the approximation quality of $\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)$ by the basis $(\phi_\ell)_{\ell \ge 1}$. We remark first that we have $|\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)| \le 2\bar{h}(t)$: this implies that for all $\ell$, and all $\theta$ we have $|e_\ell^*(\theta)| \le 2\bar{h}_\ell$ with $\bar{h}_\ell = \langle \bar{h}, \varphi_\ell \rangle$. The global rate of convergence of the series $\sum_\ell e_\ell^2(\theta)$ is controlled uniformly by the rate of $\sum_{\ell \ge 1} H_\ell^2$, denoted $r_L^2 = \sum_{\ell > L} H_\ell^2$. By orthogonality, we can write

$$ \| \mathbf{E}_{\mathcal{F}}(\phi^*, \theta) \|_{L^2}^2 = \| \mathbf{E}_L(\phi^*, \theta) \|_{L^2}^2 + \| \mathbf{R}_L(\phi^*, \theta) \|_{L^2}^2 $$

which means that is $\|\mathbf{E}_L(\phi^*, \theta)\|_{L^2}^2$ is a perturbation of the function $\theta \mapsto \|\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)\|_{L^2}^2$ by the function $\theta \mapsto -\|\mathbf{R}_L(\phi^*, \theta)\|_{L^2}^2$. This perturbation $\|\mathbf{R}_L(\phi^*, \theta)\|_{L^2}^2$ is uniformly dominated by $r_L$, hence it becomes possible to relate the two minima.

From assumption C5, we know $\|\mathbf{R}_L(\phi^*, \theta)\|_{L^2}^2$ is differentiable and we compute a series decomposition of its gradient thanks to $\partial_\theta \mathbf{R}_L(\phi^*, \theta) = \sum_{\ell > L} \partial_\theta e_\ell(\phi^*, \theta) \varphi_\ell$:

$$ \partial_\theta \| \mathbf{R}_L(\phi^*, \theta) \|_{L^2}^2 = 2 \mathbf{R}_L(\phi^*, \theta) \partial_\theta \mathbf{R}_L(\phi^*, \theta). $$

and we recall that $\mathbf{R}_L(\phi^*, \theta)$ converges uniformly to 0 and $\partial_\theta \mathbf{R}_L(\phi^*, \theta)$ is uniformly bounded in $\theta$ on $\Theta$ (as a continuous function on the compact set $\Theta$). Starting from this last remark, we use the Implicit Function Theorem to the continuously differentiable function $G(\epsilon, \theta) = \partial_\theta \|\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)\|_{L^2}^2 - 2\epsilon \partial_\theta \mathbf{R}_L(\phi^*, \theta)$, in order to get a Taylor expansion. We denote $\theta_\epsilon$ the solution to $G(\epsilon, \theta_\epsilon)$, and we remark that in particular $G(0, \theta^*) = 0$. Thus, there exists $\epsilon_0, \delta_0 > 0$ and a function $\psi: ]-\epsilon_0, \epsilon_0[ \to B(\theta^*, \delta_0)$ such that $\psi(\epsilon) = \theta_\epsilon$ (i.e. $G(\epsilon, \psi(\epsilon)) = 0$). We can also compute the first order variation of $\psi: \psi(\epsilon) = \psi(0) + \epsilon\psi'(0) + o(\epsilon)$ where

$$ \psi'(0) = -2 (\partial_{\theta\theta} \| \mathbf{E}_{\mathcal{F}}(\phi^*, \theta) \|_{L^2}^2)^{-1} \partial_{\theta} \mathbf{R}_L(\phi^*, \theta^*). $$

Since $\mathbf{R}_L(\phi^*, \theta)$ converges uniformly to 0, there exists $L_0 > 0$, such that for $L > L_0$, $|\mathbf{R}_L(\phi^*, \theta)| \le \epsilon_0$ so that we can apply the linearization above to $G(\mathbf{R}_L(\phi^*, \theta), \theta) = \partial_\theta \|\mathbf{E}_{\mathcal{F}}(\phi^*, \theta)\|_{L^2}^2 - \partial_\theta \|\mathbf{R}_L(\phi^*, \theta)\|_{L^2}^2$.
<table><thead><tr><th></th><th>θ̂<sup>OC</sup><sub>3</sub></th><th>θ̂<sup>OC</sup><sub>4</sub></th><th>θ̂<sup>NLS</sup></th><th>θ̂<sup>ref</sup></th></tr></thead><tbody><tr><th>ρ<sub>m</sub></th><td>2.9e-5</td><td>2.7e-5</td><td>1.5e-5</td><td>1.6e-5</td></tr><tr><th>ρ<sub>s</sub></th><td>4.1e-5</td><td>4.7e-5</td><td>4.1e-5</td><td>4.5e-5</td></tr><tr><th>δ<sub>l</sub></th><td>2.0</td><td>3.4</td><td>3.7</td><td>3.96</td></tr><tr><th>γ<sub>ms</sub></th><td>0.39</td><td>0.35</td><td>0.15</td><td>0.157</td></tr><tr><th>γ<sub>sl</sub></th><td>0.72</td><td>0.81</td><td>0.47</td><td>0.49</td></tr><tr><th>RMSE</th><td>13.5</td><td>13.9</td><td>9.0</td><td>9.5</td></tr></tbody></table>

<table><caption>Table 6.1: Estimates, RMSE and the 95% confidence intervals for different L and estimators.</caption><thead><tr><th></th><th colspan="2">θ̂<sup>OC</sup><sub>3</sub></th><th colspan="2">θ̂<sup>OC</sup><sub>4</sub></th><th colspan="2">θ̂<sup>NLS</sup></th></tr><tr><th></th><th>Low. Bound</th><th>Up. Bound</th><th>Low. Bound</th><th>Up. Bound</th><th>Low. Bound</th><th>Up. Bound</th></tr></thead><tbody><tr><th>ρ<sub>m</sub></th><td>2.1e-5</td><td>3.7e-5</td><td>1.9e-5</td><td>3.4e-5</td><td>0.7e-0.5</td><td>2.4e-0.5</td></tr><tr><th>ρ<sub>s</sub></th><td>0.7e-5</td><td>7.4e-5</td><td>0.9e-5</td><td>8.4e-5</td><td>3.4e-0.5</td><td>4.8e-0.5</td></tr><tr><th>δ<sub>l</sub></th><td>-1.11</td><td>5.21</td><td>-0.28</td><td>7.21</td><td>2.59</td><td>4.93</td></tr><tr><th>γ<sub>ms</sub></th><td>0.27</td><td>0.50</td><td>0.24</td><td>0.46</td><td>0.03</td><td>0.26</td></tr><tr><th>γ<sub>sl</sub></th><td>-0.10</td><td>1.55</td><td>-0.14</td><td>1.76</td><td>0.39</td><td>0.55</td></tr></tbody></table>

1

<figure><img src="image_4.png" /><figcaption>Figure 6.1: Influenza model, Estimated curves for $X_1$ (red), $X_2$ (green), $X_3$ (blue); ×: observations, □: solution for $\hat{\theta}_1^{OC}$, ◦: solution for $\hat{\theta}_2^{OC}$, solid line: solution with $\hat{\theta}^{NLS}$.</figcaption></figure>

2

3
1 We obtain that the minima $\theta_L^*$ and $\theta^*$ are such that

$$ \theta_L^* = \theta^* + \mathbf{R}_L(\phi^*, \theta)2 (\partial_{\theta\theta} \|\mathbf{E}_F(\phi^*, \theta)\|_{L^2}^2)^{-1} \partial_{\theta}\mathbf{R}_L(\phi^*, \theta^*) + o(r_L). $$

2 This implies that $|\theta_L^* - \theta^*| = O(r_L)$. If we use the sine basis, this means that $\mathbf{P}_F H = \sum_{\ell} \bar{h}_{\ell} \sqrt{2} \sin(\pi \ell t)$

3 but it is also in $H_0^1$ (as it is in $H^1$), then we have $\sum_{\ell \ge 1} \ell^2 \bar{h}_{\ell}^2$ and $r_L^2 = o(\frac{1}{L^2})$. $\square$

4 Proposition 3

5 If conditions C1-C6 are satisfied, then

$$ \left[ \mathbf{J}_L (\hat{\phi}, \hat{\theta}_{n,L})^{\top} \tilde{\mathbf{J}}_L \right]^{-1} \mathbf{J}_L (\hat{\phi}, \hat{\theta}_{n,L})^{\top} \xrightarrow{P} \mathbf{M}_L^* = \left[ \mathbf{J}_{\theta,L}^{*\top} \mathbf{J}_{\theta,L}^{*} \right]^{-1} \mathbf{J}_{\theta,L}^{*\top} \quad (A.3) $$

6 and we have

$$ \hat{\theta}_{n,L} - \theta_L^* = -\mathbf{M}_L^* \mathbf{e}_L(\hat{\phi}, \theta^*) + o_P(1). \quad (A.4) $$

7 The first order condition implies that $\hat{\theta}_{n,L}$ satisfies (4.1). We develop a Taylor expansion of $f(t, \hat{\phi}(t), \theta)$

8 around $\theta^*$ of order 1 for $t$ a.e. in $[0, 1]$: $e_\ell(\hat{\phi}, \theta)$ as $e_\ell(\hat{\phi}, \theta^*) + \langle f_\theta(\cdot, \hat{\phi}, \tilde{\theta}), \varphi_\ell \rangle$ where $\tilde{\theta}$ is on straight

9 line between $\theta^*$ and $\hat{\theta}_{n,L}$. We can write it in vector form

$$ \mathbf{e}_L(\hat{\phi}, \hat{\theta}_{n,L}) = \mathbf{e}_L(\hat{\phi}, \theta^*) + \tilde{\mathbf{J}}_{\theta,L}(\hat{\theta}_{n,L} - \theta_L^*). \quad (A.5) $$

10 $\tilde{\mathbf{J}}_{\theta,L}$ is a matrix $\mathbb{R}^{L \times p}$ with entries $\langle f_{\theta_i}(\cdot, \hat{\phi}, \tilde{\theta}), \varphi_\ell \rangle$. Thus, if we premultiply (A.5) by $\mathbf{J}_{\theta,L}(\hat{\phi}, \hat{\theta}_{n,L})^\top$, we get the asymptotic expansion

$$ 0 = \mathbf{J}_{\theta,L} (\hat{\phi}, \hat{\theta}_{n,L})^{\top} \mathbf{e}_L(\hat{\phi}, \theta^*) + \mathbf{J}_{\theta,L} (\hat{\phi}, \hat{\theta}_{n,L})^{\top} \tilde{\mathbf{J}}_{\theta,L} (\hat{\theta}_{n,L} - \theta_L^*). \quad (A.6) $$

12 This shows that the key results for relating the behavior of $(\hat{\theta}_{n,L} - \theta_L^*)$ to the behavior of $\mathbf{e}_L(\hat{\phi}, \theta^*)$ is

13 the convergence in probability of $\mathbf{J}_{\theta,L}(\hat{\phi}, \hat{\theta}_{n,L})^\top$. Indeed, if of to prove the convergence of true since the
<figure><img src="image_5.png" /><figcaption>Figure 6.2: Influenza model, Estimated curves for $X_1$ (red), $X_2$ (green), $X_3$ (blue); □ solution obtained with OC+NLS, ◦ solution obtained with $\tilde{\theta}^{ref}$.</figcaption></figure>

## 6.2 Blowfly model

The Delay Differential Equation (1.3) was proposed by Gurney et al [40] to model the dynamics of a population of blowflies, from the Nicholson's blowfly data [26]. These data consists of 350 counts taken every two days during between day 40 = $T_0$ and day 315 = $T_1$. As Gurney did, we take $\tau = 14.8$ days and our aim is to estimate $\theta = (P, N_0, \delta)$. The orthogonal conditions derived from the weak form is $\forall \varphi \in C_c^1([a, b])$,

$$ \int_a^b N(u) \dot{\varphi}(u) du + P \int_{a-\tau}^{b-\tau} N(u) e^{-\frac{N(u)}{N_0}} \varphi(u+\tau) du - \delta \int_a^b N(u) \varphi(u) du = 0 $$

where $[a, b]$ has to be chosen such that: $[a, b]$, $[a - \tau, b - \tau] \subset [T_0, T_1]$. Due to a change in the dynamics, we have used only the first 180 observations, see [33]. For the nonparametric estimation, we have used 42 knots located between $t = 40$ and $t = 220$. Preliminary tests and comparisons suggests to use the sine basis for the test function $\varphi_\ell$, and we use $2 \le L \le 15$. A simulation is given in figure 6.3

11

12
1 matrix $\mathbf{J}_L^* \mathbf{J}_L^*$ is nonsingular and

$$ \left[ \mathbf{J}_{\theta,L} (\hat{\phi}, \hat{\theta}_{n,L})^{\top} \tilde{\mathbf{J}}_L \right]^{-1} \mathbf{J}_{n,L} (\hat{\theta}_{n,L})^{\top} $$

2 converges in probability to $[\mathbf{J}^*\mathbf{J}^*]^{-1} \mathbf{J}^* = \mathbf{M}$. Indeed, $\mathbf{J}_{n,L}(\theta)$ converges uniformly to $\mathbf{J}_L(\theta)$ because the

3 functions $\frac{\partial}{\partial \theta_i} f$ are uniformly Lipschitz, thus this implies that $\hat{\theta}_{n,L} - \theta_L^* = -\mathbf{M}_L \mathbf{e}_{n,L}(\theta^*) + o_P(1)$.

4 References

5 [1] D. K. Andrews. Asymptotic normality of series estimators for nonparametric and semiparametric regression models. *Econometrica*, 59(2):307–345, 1991.

7 [2] P.J. Bickel and Y. Ritov. Nonparametric estimators which can be plugged-in. *Annals of Statistics*, 31(4):4, 2003.

9 [3] S. C. Brenner and L. R. Scott. *The Mathematical Theory of Finite Element Methods*. Springer, 2008.

11 [4] N. J-B. Brunel. Parameter estimation of ode’s via nonparametric estimators. *Electronic Journal of Statistics*, 2:1242–1267, 2008.

13 [5] O. Cappé, E. Moulines, and T. Rydén. *Inference in Hidden Markov Models*. Springer-Verlag, 2005. Localisation : Nicolas.

15 [6] Hein W Engl, Christoph Flamm, Philipp Kügler, James Lu, Stefan Müller, and Peter Schuster. Inverse problems in systems biology. *Inverse Problems*, 25(12), 2009.

17 [7] L. Goldstein and K. Messer. Optimal plug-in estimators for nonparametric functional estimation. *The annals of statistics*, 20(3):1306–1328, 1992.

19 [8] S. Gugushvili and C.A.J. Klaassen. Root-n-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing. *Bernoulli*, to appear, 2011.

21 [9] A.R. Hall. *Generalized Method of Moments*. Oxford University Press, 2005.
<table><thead><tr><th></th><th>L = 11</th><th>L = 9</th><th>L = 12</th></tr></thead><tbody><tr><th>P</th><td>7.81</td><td>7.52</td><td>7.91</td></tr><tr><th>N<sub>0</sub></th><td>381.8</td><td>385.9</td><td>377.7</td></tr><tr><th>δ</th><td>0.154</td><td>0.153</td><td>0.154</td></tr><tr><th>RSSE</th><td>1.7136e+03</td><td>1.7557e+03</td><td>1.7990e+03</td></tr></tbody></table>

<table><caption>Table 6.2: Estimates, RSSE and 95% confidence intervals for different L</caption><thead><tr><th></th><th colspan="2">L = 11</th><th colspan="2">L = 9</th><th colspan="2">L = 12</th></tr><tr><th>O.C</th><th>Low. Bound</th><th>Up. Bound</th><th>Low. Bound</th><th>Up. Bound</th><th>Low. Bound</th><th>Up. Bound</th></tr></thead><tbody><tr><th>P</th><td>5.80</td><td>9.81</td><td>5.64</td><td>9.40</td><td>5.0416</td><td>10.77</td></tr><tr><th>N<sub>0</sub></th><td>303.62</td><td>459.94</td><td>306.59</td><td>465.38</td><td>289.36</td><td>465.98</td></tr><tr><th>δ</th><td>0.10</td><td>0.20</td><td>0.11</td><td>0.19</td><td>0.10</td><td>0.20</td></tr></tbody></table>

# 1 7 Discussion

Among the simulated models we considered (α-pinene, Ricatti), the NLS estimator is often the best estimator in the asymptotic case (and small noise case) in terms of MSE for the parameters. Nevertheless, in some complex case such as unknown initial conditions for α-pinene (with small sample size or high noise level), or Ricatti equation (with known or unknown change point $T_r$), then TS and OC can offer better statistical performances. The α-pinene model shows the interest of using information on the boundaries in OC (as introduced in section 2.1). Moreover, simulations show that OC can improve on classical TS although it uses only (partial information) about (weak) derivatives. The fact that the NLS can be caught up, even in the very favorable case of a closed-form solution and starting values (for NLS optimization) close to the true parameter indicates that the introduction of Functions Moments offers a competitive estimator to the direct classical for complex case. In the latter case of Ricatti, the TS approaches is uniformly better than NLS, whereas OC is not systematically better than NLS. Ricatti Equation is striking, as it shows that good proxies $\hat{\phi}$ gives a lot of information: when $T_r$ is known, the reconstruction of the solution and its derivative is excellent, which gives a clear advantage to the plain TS. Nevertheless, when $T_r$ is unknown the derivative estimation is of poor quality around $T_r$, and the TS estimator is unstable and cannot be computed. The same situation occurs for NLS, because of some lack of identifiability and dramatic changes in derivative estimation which makes the optimization algorithms inefficient. For the influenza dataset analysis, the two OC estimators give correct parameter estimates from real and sparse data (the simulated ODE have a correct qualitative behavior). When used as starting for NLS, both estimates give the same NLS estimator, which improves (obviously) the
1

2

[10] E. L Ionides, A. Bhadra, Y. Atchade, and A. A King. Iterated filtering. *Annals of Statistics*, 39:1776–1802, 2011.

3

4

[11] E. L. Ionides, C. Breto, and A. A. King. Inference for nonlinear dynamical systems. *Proceedings of the National Academy of Sciences*, 103:18438–18443, 2006.

5

6

7

[12] H Liang and H. Wu. Parameter estimation for differential equation models using a framework of measurement error in regression models. *Journal of the American Statistical Association*, 103(484):1570–1583, December 2008.

8

9

10

[13] J. Madar, J. Abonyi, H. Roubos, and F. Szeifert. Incorporating prior knowledge in cubic spline approximation - application to the identification of reaction kinetic models. *Industrial and Engineering Chemistry Research*, 42(17):4043–4049, 2003.

11

12

[14] H. Miao, X. Xia, A. S. Perelson, and H. Wu. On identifiability of nonlinear ode models and applications in viral dynamics. *SIAM Review*, 53:3–39, 2011.

13

14

[15] C.G. Moles, P. Mendes, and J.R. Banga. Parameter estimation in biochemical pathways: a comparison of global optimization methods. *Genome Research*, 13:2467–2474, 2003.

15

16

[16] W. K. Newey. Convergence rates and asymptotic normality for series estimators. *Journal of Econometrics*, 79:147–168, 1997.

17

18

[17] D. O'Regan. *Existence theory for nonlinear ordinary differential equations*. Mathematics and its applications. Kluwer, 1997.

19

20

21

[18] A. A. Poyton, M.S. Varziri, K.B. McAuley, P.J. McLellan, and J.O. Ramsay. Parameter estimation in continuous-time dynamic models using principal differential analysis. *Computers and Chemical Engineering*, 30:698–708, 2006.

22

23

[19] L. Pronzato. Optimal experimental design and some related control problems. *Automatica*, 44:303–325, 2008.
<figure><img src="image_6.png" /><figcaption>Figure 6.3: Solution *N* of the Nicholson’s DDE simulated with the OC estimator (computed with *L* = 11 conditions - continuous red line). The NLS solution is given by the dashed green curve. The initial function is estimated between day 40 and 55 and the simulation starts after day 55. Drift between data and simulations comes from a chaotic behavior and uncertainty in initial condition (and parameters).</figcaption></figure>

1 SSE and still gives an estimator closer to the estimates given Wu et al (and same qualitative behavior for the solution). We consider the (self-)consistency of the OC estimates as an indication for reliability of the OC approach. More generally, OC can be used for initializing a NLS estimator, which is often a critical problem in nonlinear regression. In our case, we found a slightly better estimate (for RSS) w.r.t the original paper by Wu et al. For the Delay Differential Equation modeling the blowfly dataset, we insist on the ease of implementation of the method, that avoids the semiparametric estimation of the initial condition. Moreover it provides an estimate close to the posterior mean obtained by ABC: *P*<sup>ABC</sup> = 7.39, *N*<sub>0</sub><sup>ABC</sup> = 365.03 and δ<sup>ABC</sup> = 0.15. With a varying number of Orthogonal Conditional, we can assess the self-consistency of our estimate. Moreover, the posterior mean is always in the 95% confidence set computed for OC.
[20] M. Quach, N. Brunel, and F. d'Alche Buc. Estimating parameters and hidden variables in non-linear state-space models based on odes for biological networks inference. *Bioinformatics*, 23(23):3209-3216, 2007.

[21] J.O. Ramsay, G. Hooker, J. Cao, and D. Campbell. Parameter estimation for differential equations: A generalized smoothing approach. *Journal of the Royal Statistical Society (B)*, 69:741-796, 2007. To appear.

[22] D. Ruppert, M.P. Wand, and R.J. Carroll. *Semiparametric regression*. Cambridge series on statistical and probabilistic mathematics. Cambridge University Press, 2003.

[23] S. van de Geer. *Empirical processes in M-estimation*. Cambridge University Press, 2000.

[24] A.W. van der Vaart. *Asymptotic Statistics*. Cambridge Series in Statistical and Probabilities for Mathematical Statistics. Cambridge University Press, 1998.

[25] J. M. Varah. A spline least squares method for numerical parameter estimation in differential equations. *SIAM J.sci. Stat. Comput.*, 3(1):28-46, 1982.

[26] E.O. Voit and J. Almeida. Decoupling dynamical systems for pathway identification from metabolic profiles. *Bioinformatics*, 20(11):1670-1681, 2004.

[27] H. Xue, H. Miao, and H. Wu. Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. *Annals of Statistics*, 38(4):2351-2387, 2010.
1 Acknowledgments

2 This work was funded by two ANR projects GD2S (ANR-05-MMSA-0013-01) and ODESSA (ANR-

3 09-SYSC-009-01) and received a partial support from the Analysis of Object Oriented Data program

4 (2010-2011) in Statistical and Applied Mathematical Sciences Institute (SAMSI), USA.

5 References

6 [1] D. K. Andrews. Asymptotic normality of series estimators for nonparametric and semiparametric regression models. *Econometrica*, 59(2):307–345, 1991.

8 [2] U.M. Ascher, R.M.M. Mattheij, and R.D. Russell. *Numerical Solutions of Boundary Value Problems for Ordinary Differential Equations*, volume 478. Prentice Hall New Jersey, 1988.

10 [3] R. Bellman. A note on the identification of linear systems. *Proc. Amer. Math. Soc.*, 17:68–71, 1966.

11 [4] R. Bellman, H. Kagiwada, and R. Kalaba. Orbit determination as a multi-point boundary-value problem and quasilinearization. *PNAS*, 48:1327–1329, 1962.

13 [5] P.J. Bickel and Y. Ritov. Nonparametric estimators which can be plugged-in. *Annals of Statistics*, 31(4):4, 2003.

15 [6] S. C. Brenner and L. R. Scott. *The Mathematical Theory of Finite Element Methods*. Springer, 2008.

17 [7] D. Brigo and F. Mercurio. *Interest Rate Models - Theory and Practice*. Springer Finance. Springer-Verlag, 2nd edition edition, 2006.

19 [8] N. J-B. Brunel. Parameter estimation of ode's via nonparametric estimators. *Electronic Journal of Statistics*, 2:1242–1267, 2008.

21 [9] O. Cappé, E. Moulines, and T. Rydén. *Inference in Hidden Markov Models*. Springer-Verlag, 2005.

22 [10] J.B. Conway. *A course in functional analysis*, volume 96. Springer, 1990.
[11] S.P. Ellner and J. Guckenheimer. *Dynamic Models in Biology*. Number vol. 13 in Princeton Paperbacks. Princeton University Press, 2006.

[12] Hein W Engl, Christoph Flamm, Philipp Kügler, James Lu, Stefan Müller, and Peter Schuster. Inverse problems in systems biology. *Inverse Problems*, 25(12), 2009.

[13] L. Goldstein and K. Messer. Optimal plug-in estimators for nonparametric functional estimation. *The annals of statistics*, 20(3):1306–1328, 1992.

[14] S. Gugushvili and C.A.J. Klaassen. Root-n-consistent parameter estimation for systems of ordinary differential equations: bypassing numerical integration via smoothing. *Bernoulli*, to appear, 2011.

[15] Ryan N Gutenkunst, Joshua J Waterfall, Fergal P Casey, Kevin S Brown, Christopher R Myers, and James P Sethna. Universally sloppy parameter sensitivities in systems biology models. *PLoS Comput Biol*, 3(10):e189, 10 2007.

[16] E. L Ionides, A. Bhadra, Y. Atchade, and A. A King. Iterated filtering. *Annals of Statistics*, 39:1776–1802, 2011.

[17] E. L. Ionides, C. Breto, and A. A. King. Inference for nonlinear dynamical systems. *Proceedings of the National Academy of Sciences*, 103:18438–18443, 2006.

[18] G.M. James. Curve alignment by moments. *Annals of Applied Statistics*, 1(2):480–501, 2007.

[19] H Liang and H. Wu. Parameter estimation for differential equation models using a framework of measurement error in regression models. *Journal of the American Statistical Association*, 103(484):1570–1583, December 2008.

[20] J. Madar, J. Abonyi, H. Roubos, and F. Szeifert. Incorporating prior knowledge in cubic spline approximation - application to the identification of reaction kinetic models. *Industrial and Engineering Chemistry Research*, 42(17):4043–4049, 2003.

[21] H. Miao, X. Xia, A. S. Perelson, and H. Wu. On identifiability of nonlinear ode models and applications in viral dynamics. *SIAM Review*, 53:3–39, 2011.
1

C.G. Moles, P. Mendes, and J.R. Banga. Parameter estimation in biochemical pathways: a comparison of global optimization methods. *Genome Research*, 13:2467-2474, 2003.

2

3

[23] B. Murray. *Mathematical Biology, Vol. 1: An Introduction, 3E*. Springer (India) Pvt. Ltd., 2004.

4

W. K. Newey. Convergence rates and asymptotic normality for series estimators. *Journal of Econometrics*, 79:147-168, 1997.

5

6

W.K. Newey and T.M. Stoker. Efficiency of weighted average derivative estimators and index models. *Econometrica*, 61(5):1199-1223, 1993.

7

8

A. J. Nicholson. The self-adjustement of population to change. *Cold Spring Harbor Symposia on Quantitative Biology*, 22:153-173, 1957.

9

10

T. Ojika and W. Welsh. A numerical method for the solution of multi-point problems for ordinary differential equations with integral constraints. *Journal of Mathematical Analysis and Applications*, 72:500-511, 1979.

11

12

13

A. A. Poyton, M.S. Varziri, K.B. McAuley, P.J. McLellan, and J.O. Ramsay. Parameter estimation in continuous-time dynamic models using principal differential analysis. *Computers and Chemical Engineering*, 30:698-708, 2006.

14

15

16

M. Quach, N. Brunel, and F. d'Alche Buc. Estimating parameters and hidden variables in non-linear state-space models based on odes for biological networks inference. *Bioinformatics*, 23(23):3209-3216, 2007.

17

18

19

J.O. Ramsay, G. Hooker, J. Cao, and D. Campbell. Parameter estimation for differential equations: A generalized smoothing approach. *Journal of the Royal Statistical Society (B)*, 69:741-796, 2007. To appear.

20

21

22

M. Rodriguez-Fernandez, J.A. Egea, and J. R Banga. Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. *BioMed Central*, 2006.

23

24

D. Ruppert, M.P. Wand, and R.J. Carroll. *Semiparametric regression*. Cambridge series on statistical and probabilistic mathematics. Cambridge University Press, 2003.

25
[33] Y. Seifu S. P. Ellner and R.H. Smith. Fitting Population Dynamic Models To Time-Series Data By Gradient Matching. *Ecology*, 83(8):2256-2270, 2002.

[34] L. Schumaker. *Spline Functions: Basic Theory*. Cambridge University Press, 3rd edition, 2007.

[35] E. Sontag. *Mathematical Control Theory: Deterministic finite-dimensional systems*. Springer-Verlag (New-York), 1998.

[36] S. van de Geer. *Empirical processes in M-estimation*. Cambridge University Press, 2000.

[37] A.W. van der Vaart. *Asymptotic Statistics*. Cambridge Series in Statistical and Probabilities Mathematics. Cambridge University Press, 1998.

[38] J. M. Varah. A spline least squares method for numerical parameter estimation in differential equations. *SIAM J.sci. Stat. Comput.*, 3(1):28-46, 1982.

[39] E.O. Voit and J. Almeida. Decoupling dynamical systems for pathway identification from metabolic profiles. *Bioinformatics*, 20(11):1670-1681, 2004.

[40] S. P. Bkythe, W. S. C. Gurney and R.L. Nisbet. Nicholson's blowflies revisited. *Nature*, 287:17-21, 1980.

[41] S.N. Wood. Statistical inference for noisy nonlinear ecological dynamic systems. *Nature*, 466(7310):1102-1104, 2010.

[42] H. Wu, A. Kumar, H. Miao, J. Holden-Wiltse, T.R. Mosmann, A.M. Livingstone, G.T. Belz, A.S. Perelson, M.S. Zand, and D.J Topham. Modeling of influenza-specific cd8+ t cells during the primary response indicates that the spleen is a major source of effectors. *The Journal Of Immunology*, 187(9):4474-4482, 2011.

[43] H. Xue, H. Miao, and H. Wu. Sieve estimation of constant and time-varying coefficients in nonlinear ordinary differential equation models by considering both numerical error and measurement error. *Annals of Statistics*, 38(4):2351-2387, 2010.