# A Trajectory Optimisation-Based Incremental Learning Strategy for Learning from Demonstration
## 1 Introduction  
Industrial robots have been designed to perform repetitive tasks, so they have been extensively used in the mass production of automotive, aerospace, and marine industries. As the demand for customised production has dramatically increased in recent years, collaborative robots (cobots) have been actively developed to carry out customised tasks within unstructured environments [1]. Learning from demonstration (LfD) is an effective approach to facilitating cobots to fulfil the above purpose. LfD can enable cobots to efficiently realise customised operations based on human demonstrations without the need for detailed motion programming or reprogramming [2, 3]. The process of LfD is illustrated in Figure 1, where a cobot performs a pick-and-place task. To create a demonstration, ahuman operator drags the end-effector of the cobot to pick up an object from a start point (the yellow box in Figure 1) and place it at a target point (the marker in Figure 1). The demonstration (trajectory) is shown as the red curve in Figure 1. The demonstration is recorded and used to train an LfD-based learning model. To accomplish a similar task where the marker is moved to a new position, the trained LfD-based learning model can adaptively generate a new trajectory (generated trajectory). This is illustrated as the blue dashed line in Figure 1.  
Dynamic movement primitive (DMP) is an effective learning model for implementing LfD [4]. Unlike Gaussian mixture modelling (GMM), the DMP model only needs to learn from one demonstration, thereby minimising the difficulty of creating multiple demonstrations [5; 6; 7]. The mathematical representation of the DMP model includes a second-order spring damping system and a force item [8]. The former ensures that the generated trajectory created using the DMP model can converge to the target without divergence. The latter, which is constructed based on a series of radial basis function (RBF) kernels, is used to control the shape of the generated trajectory and manage the convergence process. The advantage of the DMP model is its robustness to perturbations [9]. However, the model exhibits some limitations as well. For instance, when the position of the marker illustrated in Figure 1 is moved far away from the previous demonstration area, the generated trajectory via DMP cannot work properly [10]. That is, the difference between the positions of the generated trajectory and the marker should not exceed \(\delta\), which is a threshold representing the maximum error allowable for the successful execution of the pick-and-place task. The situation is illustrated by the green dashed curve in Figure 1. To address the issue, the following considerations are given:  
* A straightforward idea is to create a new demonstration for the new situation. However, redemonstration is time consuming as the experimental environment needs to be set up again [3]. To mitigate the problem, it will be beneficial to reuse part of the originally created demonstration to adaptively generate a new trajectory (namely, an extended demonstration) for the new target;
* DMP is a one-shot learning model, which means that when the extended demonstration is learned, the knowledge gained from the previous demonstration is forgotten. Therefore, it is imperative to develop an incremental learning mechanism for the DMP model to improve its generalisation capability for various situations. Various approaches for trajectory generation were developed to complement demonstration data, such as task-parameterised Gaussian mixture models (TP-GMMs) [3], adversarial generative models [11], B-spline-based models [12], and PRM-based models [13]. These approaches can generate new trajectories flexibly. However, the lack of imitation and utilisation of prior knowledge (i.e., existing demonstrations) poses safety risks in LfD applications. In addition, some researchers attempted to improve the performance of DMP [14].