## 9 Convergence of the Learning Procedure  
In this section, we prove the convergence of the learning procedure in a simplified version similar to that described in Section 2.2, but without the discretization with respect to \(r\). Then, we shall discuss additional conditions for the convergence of its discretized version. Let \(X(r),X_{1}(r),\ldots,X_{n}\) be independent, identically distributed random variables with parameter \(r\), with p.d.f. \(f(.,r)\in L^{2}([R_{1},\,R_{2}])\), where--for simplicity--we assume that \(n\) is the same for each \(r\). Then, \(f\) has the representation  
\[f(x,\,r)=\sum_{k=1}^{\infty}a_{k}(r)\,v_{k}(x), \tag{52}\]  
which is convergent in the \(L^{2}([S_{1},\,S_{2}])\) norm, where  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]  
Then, we estimate \(a_{k}(r)\)'s as follows:  
\[\hat{a}_{k}(r)=\frac{1}{n}\sum_{i=1}^{n}v_{k}(X_{i}(r)). \tag{54}\]  
**Lemma 4**: _For every \(r\in[R_{1},\,R_{2}]\)\(\hat{a}_{k}(r)\) is the unbiased estimator of \(a_{k}(r)\)._  
Proof. : Indeed,  
\[E_{r}(\hat{a}_{k}(r))=\frac{1}{n}\sum_{i=1}^{n}\,E_{r}[v_{k}(X_{i}(r))]=a_{k}( r). \tag{55}\]  
As an estimator of \(f(.,r)\), we take the truncated version of (52):  
\[\tilde{f}_{n}(x,r)=\sum_{k=1}^{K(n)}\hat{a}_{k}(r)\,v_{k}(x), \tag{56}\]  
Figure 10: A result of smoothed simulation for 50k rpm. where the truncation point \(K\) depends on \(n\). It may also depend on \(r\), but, for the asymptotic analysis, we take this simpler version. The standard way of assessing the distance between \(f\) and its estimator \(\tilde{f}_{n}\) is the mean integrated squared error (\(MISE\)). Notice, however, that, in our case, the MISE additionally depends on \(r\), which is further denoted as \(MISE(r)\). Thus, in order to have a global error, we consider the integrated \(MISE(r)\), which is defined as follows:  
\[IMISE_{n}\ =\ \int_{R_{1}}^{R_{2}}MISE_{n}(r)\,dr. \tag{57}\]  
The \(MISE(r)\) is defined as follows:  
\[MISE_{n}(r)=E_{r}\int_{S_{1}}^{S_{2}}[f(x,\ r)-\tilde{f}_{n}(x,\ r)]^{2}dx, \tag{58}\]  
where the expectation w.r.t. \(f(.,\ r)\) concerns all \(X_{i}(r)\)'s that are present in (56). Using the orthonormality of \(v_{k}\)'s, we obtain:  
\[MISE_{n}(r)=E_{r}\int_{S_{1}}^{S_{2}}[\sum_{k=1}^{K(n)}(a_{k}(r)-\hat{a}_{k}(r ))^{2}\,v_{k}(x)-\sum_{k=K(n)+1}^{\infty}a_{k}(r)\,v_{k}(x)  
For evaluating the variance component, we use Lemma 1:  
\[Var(K(n),\,r)=\sum_{k=1}^{K(n)}\,E_{r}[(a_{k}(r)-\hat{a}_{k}(r))^{2}]=\sum_{k=1}^{ K(n)}\frac{E_{r}(\hat{a}_{k}(r)-E_{r}\hat{a}_{k}(r))^{2}}{Var_{r}(\hat{a}_{k}(r))}, \tag{65}\]  
where \(Var_{r}(. )\) is the variance of a r.v. having the p.d.f. \(F(x,\,r)\). Let us assume that the orthonormal and complete system \(v_{k}\)'s is uniformly bounded, i.e., there exists \(p\) being a nonnegative integer and \(C_{p}\) such that  
\[\sup_{x\in[S_{1},S_{2}]}|v_{k}(x)|\,\leq\,C_{p}\,k^{p},\quad k=1,\,2,\,\ldots\,. \tag{66}\]  
Notice that (66) holds for the trigonometric system with \(p=0\)  
**Lemma 6**. : _If (66) holds, then_  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]0  
Proof. : \(X_{i}(r)\)'s are i.i.d. and (66) holds. Thus,  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]1  
Using this fact in (65) finishes the proof. Notice that the bound in (67) does not depend on \(r\). **Theorem 2**. : _Let the assumptions of Lemmas 5 and 6 hold. If the sequence \(K(n)\) is selected in such a way that the following conditions hold:_  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]2  
_then the estimation error \(IMISE_{n}\to 0\,\,\text{as}\,\,n\rightarrow\infty\)._  
Proof. : By Lemmas 2 and 6, we have uniform (in \(r\)) bounds for the variance and for the squared bias, respectively. Thus,  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]3  
Hence,  
\[a_{k}(r)=\int_{S_{1}}^{S_{2}}v_{k}(x)\,f(x,\,r)\,dx=E_{r}(v_{k}(X(r))). \tag{53}\]4  
which finishes the proof by applying (69).