# A Trajectory Optimisation-Based Incremental Learning Strategy for Learning from Demonstration
## 3 Research Methodology
### DMP Modelling and Incremental Learning Updating
#### 3.3.2 Bi-DMP for Incremental Learning Updating  
In this section, the structure and mathematical formulation of Bi-DMP are constructed. The core task of Bi-DMP is to construct the incremental force item for DMP. Inspired by [PERSON] et al. [18; 19] and [PERSON] et al. [20], the main structure of the BLS designed for DMP is shown in Figure 7.  
In Figure 7, the grey dots symbolise the output of the BLS. The hidden layer of the BLS is represented by blue, red, and orange dots. The blue dots denote mapping features, while the red and orange dots correspond to enhancement nodes and additional enhancement nodes, respectively. Therefore, the incremental force item is the hidden layer combined with the weight corresponding to \(\Delta\mathbf{F}_{\mathbf{I}n}\). Further explanations of the remaining labels in Figure 7 will be provided in the following sections. This paper postulates that the force item of the \(y_{d}\) is \(\mathbf{F}_{ori}\). \(\mathbf{F}_{ori}\) has been computed and parameterised by LWR [24] and is represented in Equation (12). The \(Tra_{gen}\) should  
Figure 7: The structure of BLS for DMP. combined with sub-1 to form a complete trajectory (namely \(Tra_{gen}\)-based complete demonstration) as shown in Figure 4c, namely \(y_{dn}\). It is used to calculate the original target force item \(\mathbf{F}_{target}\). Both \(\mathbf{F}_{ori}\) and \(\mathbf{F}_{target}\) are sets containing the force item value for each point in demonstrations. Replacing \(y_{d}\) with \(y_{dn}\) in Equation (10), the initial \(\mathbf{\Delta F}_{\mathbf{In}}\) can be written as Equation (15):  
\[ \begin{split}\mathbf{\Delta F}_{\mathbf{In}}&= \mathbf{F}_{target}-\mathbf{F}_{ori}\\ &=\frac{\tau^{2}\tilde{y}_{dn}-\alpha_{y_{dn}}\left(\beta_{y_{dn} }\left(\underline{s}_{y_{dn}}-y_{dn}\right)-\tau\dot{y}_{dn}\right)}{K}+ \theta\big{(}g_{y_{dn}}-y_{dn0}\big{)}-f(\vartheta)\end{split}  \tag{15}\]  
The key components of the hidden layer are \(\mathbf{Z}_{i}\), \(\mathbf{H}\), and \(\mathbf{E}\), respectively. \(\mathbf{Z}_{i}\) refers to the mapping features, \(\mathbf{H}\) represents the enhancement nodes, and \(\mathbf{E}\) indicates the additional enhancement nodes. This paper assumes that \(u\) is the number of mapping features, \(l\) denotes the number of enhancement nodes, and \(s\) indicates the number of additional enhancement nodes, as depicted from Equation (16) to Equation (18):  
\[\mathbf{Z}_{i}=\delta\Big{(}\boldsymbol{\varphi}(\vartheta)\mathbf{W}_{ \mathbf{Z}_{i}}+\beta_{\mathbf{Z}_{i}}\Big{)},i=1,2,\ldots,u \tag{16}\]  
\[\mathbf{H}=\xi\big{(}\Xi(\vartheta)\mathbf{W}_{\mathbf{H}_{i}}+\beta_{ \mathbf{H}_{i}}\big{)} \tag{17}\]  
\[\mathbf{E}=\xi(\Xi(\vartheta)\mathbf{W}_{\mathbf{E}_{s}}+\beta_{\mathbf{E}_{ s}}) \tag{18}\]  
\(\mathbf{W}_{\mathbf{Z}_{i}}\) is a randomly initialised weight matrix and subsequently fine-tuned via a sparse autoencoder [20]. Both \(\mathbf{W}_{\mathbf{H}_{i}}\) and \(\mathbf{W}_{\mathbf{E}_{s}}\) are initialised as orthogonal matrixes. The terms \(\beta_{\mathbf{Z}_{i}}\), \(\beta_{\mathbf{H}_{i}}\), and \(\beta_{\mathbf{E}_{s}}\) represent biases, while \(\delta\) and \(\xi\) are activation functions. As suggested in [20], \(\delta\) can be a linear activation function, while \(\xi\) can be the Tansig function, as detailed in Equation (19):  
\[\xi(\cdot)=\frac{2}{1+e^{-2(\cdot)}}-1 \tag{19}\]  
To determine the weight \(\mathbf{W}^{\mathbf{\Delta F}_{\mathbf{In}}}\) corresponding to \(\mathbf{\Delta F}_{\mathbf{In}}\), it is necessary to calculate the basis weight matrix \(\mathbf{W}^{b}\) based on mapping features and enhancement nodes. Equations (20)-(22) represent the computation process:  
\[ \begin{split}\mathbf{\Delta F}_{\mathbf{In}}&= \big{[}\mathbf{Z}_{1},\ldots,\mathbf{Z}_{u}\big{|}\xi\big{(}\Xi(\vartheta) \mathbf{W}_{\mathbf{H}_{i}}+\beta_{\mathbf{H}_{i}}\big{)}\big{]}\mathbf{W}^{b }\\ &=[\mathbf{Z}|\mathbf{H}]\mathbf{W}^{b}\\ &\mathbf{W}^{b}=[\mathbf{Z}|\mathbf{H}]^{-1}\mathbf{\Delta F}_{ \mathbf{In}}=\mathbf{\mathcal{A}}^{\dagger}\mathbf{\Delta F}_{\mathbf{In}} \end{split}