A.4 Proof of lemma 2
We group terms 2 by 2 in the product \Pi^{2N} . One has (I+\frac{\theta^{2N}_{2n}}{2N})(I+\frac{\theta^{2N}_{2n-1}}{2N})=(I+\frac{\tilde{\theta}^{N}_{n}}{N}) with

\tilde{\theta}_{n}^{N}=(\frac{\theta^{2N}_{2n}+\theta^{2N}_{2n-1}}{2}+\frac{\theta^{2N}_{2n}\theta^{2N}_{2n-1}}{4N}),

So that \Pi^{2N}=\tilde{\Pi}^{N} where \tilde{\Pi}^{N} is defined as \Pi^{N} with \tilde{\theta}_{n}^{N} .
One has by Prop. 2 that

\tilde{\theta}_{n}^{N}={\theta}_{2n}^{2N}+O(\frac{1}{N}).

We will show that \tilde{\theta}_{n}^{N}={\theta}_{n}^{N}+O(\frac{1}{N}).

Let D^{N}_{n}=\|\theta^{N}_{n}-\tilde{\theta}^{N}_{n}\| and D^{N}=\frac{1}{N}\sum_{n=1}^{N}D_{n}. We have

2D^{N}_{n}\dot{D}^{N}_{n}=-N\langle abla^{N}_{n}-\tilde{ abla}^{N}_{n},\theta^{N}_{n}-\tilde{\theta}^{N}_{n}\rangle.

In addition, we have

N( abla^{N}_{n}-\tilde{ abla}^{N}_{n})=\Pi_{:n}^{N\top}(\Pi^{N}-B)\Sigma\Pi_{n:}^{N\top}-\tilde{\Pi}_{:n}^{N\top}(\tilde{\Pi}^{N}-B)\Sigma\tilde{\Pi}_{n:}^{N\top}

so that

N( abla^{N}_{n}-\tilde{ abla}^{N}_{n})=(\Pi^{N}_{:n}-\tilde{\Pi}_{:n}^{N})^{\top}(\Pi^{N}-B)\Sigma\Pi_{n:}^{N\top}+\tilde{\Pi}_{:n}^{N\top}(\Pi^{N}-B)\Sigma(\Pi^{N}_{n:}-\tilde{\Pi}_{n:}^{N})^{\top}+\tilde{\Pi}_{:n}^{N\top}(\Pi^{N}-\tilde{\Pi}^{N})\Sigma\tilde{\Pi}_{n:}^{N\top}.

Note also that since the Jacobian of (\theta_{1},..,\theta_{N})\to\Pi^{N} is

J_{(\theta_{1},..,\theta_{N})}(H_{1},..,H_{N})=\frac{1}{N}\sum_{n=1}^{N}\Pi^{N}_{:n}H_{n}\Pi^{N}_{n:}

and the \theta^{N}_{n} ’s are such that \|\theta^{N}_{n}\|\leq\frac{1}{2} , there exists a constant K>0 such that \|\Pi^{N}_{:n}-\tilde{\Pi}^{N}_{:n}\|\leq KD^{N} .
Again because \|\theta^{N}_{n}\|\leq\frac{1}{2} and \|\tilde{\theta}^{N}_{n}\|\leq\frac{1}{2} , this gives

N\| abla_{n}-\tilde{ abla}_{n}\|\leq\alpha KD^{N}\sqrt{{\ell}^{N}}+\beta\|\Pi^{N}-\tilde{\Pi}^{N}\|

for some constants \alpha , \beta .
Finally, we have

\dot{D}^{N}_{n}\leq\frac{1}{2}(\alpha KD^{N}\sqrt{{\ell}^{N}}+\beta\|\Pi^{N}-\tilde{\Pi}^{N}\|)

which gives \forall t

2{D}^{N}_{n}(t)\leq\alpha K\int_{0}^{t}D^{N}\sqrt{{\ell}^{N}}+\beta\int_{0}^{t}\|\Pi^{N}-\tilde{\Pi}^{N}\|+O(\frac{1}{N})

We now focus on the \beta term involving \|\Pi^{N}-\tilde{\Pi}^{N}\| .
Denote \Delta^{N}=\Pi^{N}-\tilde{\Pi}^{N} . One has

\dot{\Delta}^{N}=-\frac{1}{N}(\sum_{n=1}^{N}\Pi^{N}_{:n}\Pi_{:n}^{N\top}(\Pi^{N}-B)\Sigma\Pi_{n:}^{N\top}\Pi^{N}_{n:}+\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N^{\top}}(\tilde{\Pi}^{N}-B)\Sigma\tilde{\Pi}_{n:}^{N\top}\tilde{\Pi}^{N}_{n:}),

and equivalently:

\dot{\Delta}^{N}=-\frac{1}{N}(\sum_{n=1}^{N}[\Pi^{N}_{:n}\Pi^{N\top}_{:n}-\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}](\Pi^{N}-B)\Sigma\Pi_{n:}^{N\top}\Pi^{N}_{n:}+\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}({\Pi^{N}}-B)\Sigma[\Pi_{n:}^{N\top}\Pi^{N}_{n:}-\tilde{\Pi}_{n:}^{N\top}\tilde{\Pi}^{N}_{n:}]+\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}({\Pi}^{N}-\tilde{\Pi}^{N})\Sigma\tilde{\Pi}_{n:}^{N\top}\tilde{\Pi}^{N}_{n:}).

Note that similarly to \|\Pi^{N}-\tilde{\Pi}^{N}\| there exist K^{\prime} such that \|\Pi^{N}_{:n}\Pi^{N\top}_{:n}-\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}\|\leq K^{\prime}D so that

\|\dot{\Delta}^{N}+\frac{1}{N}\sum_{n=1}^{N}\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}\Delta^{N}\tilde{\Pi}_{n:}^{N\top}\tilde{\Pi}^{N}_{n:}\|\leq aK^{\prime}D\sqrt{\ell^{N}}.

Let us denote by H the operator:

H(\Delta)=\frac{1}{N}\sum_{n=1}^{N}\tilde{\Pi}^{N}_{:n}\tilde{\Pi}_{:n}^{N\top}\Delta\Sigma\tilde{\Pi}_{n:}^{N\top}\tilde{\Pi}^{N}_{n:}.

Our (PL) conditions precisely write -\Delta^{\top}H(\Delta)\leq-\lambda\|\Delta\|^{2} for some \lambda>0 .
Let \varphi^{N}=\frac{1}{2}\|\Delta^{N}\|^{2}. One has

\frac{{\mathrm{d}}\varphi^{N}}{{\mathrm{d}}t}=\langle\Delta^{N},\dot{\Delta}^{N}+H(\Delta^{N})\rangle-\langle\Delta^{N},H(\Delta^{N})\rangle

so that

\frac{{\mathrm{d}}\varphi^{N}}{{\mathrm{d}}t}\leq(aK^{\prime}D\sqrt{\ell^{N}})\sqrt{2\varphi^{N}}-2\lambda\varphi^{N}.

Since \|\Delta^{N}\|=\sqrt{2\varphi^{N}} we get

\frac{{\mathrm{d}}\|\Delta^{N}\|}{{\mathrm{d}}t}=\frac{\frac{{\mathrm{d}}\varphi^{N}}{{\mathrm{d}}t}}{\sqrt{2\varphi^{N}}}.

We finally have

\frac{{\mathrm{d}}\|\Delta^{N}\|}{{\mathrm{d}}t}\leq aK^{\prime}D^{N}\sqrt{\ell^{N}}-\lambda\|\Delta^{N}\|.

Integrating, we get

\|\Delta^{N}(t)\|\leq-\lambda\int_{0}^{t}\|\Delta^{N}\|+\int_{0}^{t}aK^{\prime}D^{N}\sqrt{l^{N}}+O(\frac{1}{N})

and then

\int_{0}^{t}\|\Delta^{N}\|\leq\frac{1}{\lambda}\int_{0}^{t}aK^{\prime}D^{N}\sqrt{l^{N}}+O(\frac{1}{N}).

Plugging this into ( 10 ) leads to

0\leq 2{D}^{N}_{n}(t)\leq\alpha K\int_{0}^{t}D^{N}\sqrt{{\ell}^{N}}+\frac{\beta}{\lambda}\int_{0}^{t}aK^{\prime}D^{N}\sqrt{\ell^{N}}+O(\frac{1}{N}).

Let n(t) be such that D^{N}_{n(t)}(t)=\max_{i\in[1,N]}D^{N}_{i}(t) .
We have

0\leq 2{D}^{N}_{n(t)}(t)\leq\mu\int_{0}^{t}D^{N}_{n(\tau)}(\tau)\sqrt{{\ell}^{N}(\tau)}d\tau+O(\frac{1}{N})

for some constant \mu>0 .
And since \sqrt{{\ell}^{N}} is integrable, we get by Gronwall’s inequality that D^{N}_{n}=O(\frac{1}{N}) \forall n\in[1,N] .
We showed:

\theta^{2N}_{2n}=\theta^{N}_{n}+O(\frac{1}{N}).

∎