# Kernel Identification of Non-Linear Systems with General Structure
## 4 Hybrid/Combined Parametric-Non-Parametric Approach  
Since the special input excitations allows for fast recovering the system characteristics only in some points, additional prior knowledge is needed to extend the model for arbitrary process \(\{u_{k}\}\). In this section we assume that the transformation \(\mathcal{F}\left(\{u_{k-i}\}_{i=0}^{\infty}\right)\) belongs to the given (a priori known), finite dimensional class of systems \(F\left(\mathbf{u}_{k}^{(s)},\theta\right)\)  
\[\mathcal{F}\left(\{u_{k-i}\}_{i=0}^{\infty}\right)\subset F\left(\mathbf{u}_{ k}^{(s)},\theta\right) \tag{45}\]  
with unknown parameter vector \(\theta\). In the proposed methodology, one of the input excitations described in Section 4 is applied. The system is identified on the finite set of \(N_{0}\) representative points \(\mathbf{x}_{[1]}^{(s)},\mathbf{x}_{[2]}^{(s)},...,\mathbf{x}_{[p]}^{(s)},..., \mathbf{x}_{[N_{0}]}^{(s)}\), where \(\mathbf{x}_{[p]}^{(s)}\in R^{s}\), and \(p=1,2,...,N_{0}\). Let us denote by \(\theta^{*}\)--the true and unknown vector of system parameters. We assume that \(\theta^{*}\) is identifiable, i.e., the following property holds  
\[\mathcal{F}\left(\{u_{k-i}\}_{i=0}^{\infty}\right)=F\left(\mathbf{u}_{k}^{(s) },\theta\right)\Longleftrightarrow\theta=\theta^{*}. \tag{46}\]  
Moreover, let the quality index  
\[Q\left(\theta\right)=E\left[y_{k}-F\left(\mathbf{u}_{k}^{(s)},\theta\right) \right]^{2} \tag{47}\]  
Figure 4: Input space for \(s=2\) and \(d=2\). Figure 3: Input space for \(s=2\) and \(d=1\), (**a**) \(k\) odd, (**b**) \(k\) even. be convex for \(\theta\in\Xi\), where \(\Xi\) is some neighborhood of the true \(\theta^{*}\)  
\[\theta^{*}=\arg\min_{\theta}Q\left(\theta\right). \tag{48}\]  
The following two-step algorithm is proposed. Step 1. (non-parametric) Using the input-output observations \(\left\{\left(u_{k},y_{k}\right)\right\}_{k=1}^{N}\) for \(p=1,2,...,N_{0}\) compute the estimates  
\[\widehat{R}_{s}(\mathbf{x}_{[1]}^{(s)}),...,\widehat{R}_{s}(\mathbf{x}_{[p]}^ {(s)}),...,\widehat{R}_{s}(\mathbf{x}_{[N_{0}]}^{(s)}), \tag{49}\]  
Step 2. (parametric) Minimize the empirical version of the least squares criterion (47)  
\[\widehat{\theta}=\arg\min_{\theta}\frac{1}{N_{0}}\sum_{p=1}^{N_{0}}\left( \widehat{R}_{s}\left(\mathbf{x}_{[p]}^{(s)}\right)-F\left(\mathbf{x}_{[p]}^{ (s)},\theta\right)\right)^{2}. \tag{50}\]  
**Lemma 1**. : _Let \(F\left(\mathbf{u}_{k}^{(s)},\theta\right)\) be Lipschitz with respect to all \(u_{k-l}\)'s included in \(\mathbf{u}_{k}^{(s)}\) and all parameters in \(\theta\). If the error of non-parametric estimate behaves like \(\left\|\widehat{R}_{s}\left(\mathbf{x}_{[p]}^{(s)}\right)-F\left(\mathbf{x}_{ [p]}^{(s)},\theta^{*}\right)\right\|=O\left(N^{-\tau}\right)\) in the mean square sense, for all \(p=1,2,...,N_{0}\), then_  
\[\left\|\widehat{\theta}-\theta^{*}\right\|=O\left(N^{-\tau}\right) \tag{51}\]  
_in the parametric step 2._  
Proof. : The property (51) can be proven following the lines of the proof of Theorem 1 in [25]. **Remark 2**. : _Fulfillment of (46) and the method of non-linear optimization in (50) are strictly dependent on the specifics of the problem. In active experiment, when the input can be generated arbitrarily, appropriate selection of the points \(\left\{\mathbf{x}_{[p]}^{(s)}\right\}_{p=1}^{N_{0}}\) can significantly simplify operations in Step 2 (see example below)._  
**Example 1**.