# A Motion Capture and Imitation Learning Based Approach to Robot Control
## 3 Related Work  
In its simplest form--known as _behavioral cloning_--imitation learning is reduced to a classification or regression task. Given a set of states and actions or state transitions produced by an unknown expert function, a model (_policy_) is trained to approximate this function. Even with very small parameter counts by modern standards, when combined with artificial neural networks this method has demonstrated some success as far back as the 1980s, provided the task has simple dynamics such as keeping a motor vehicle centered on a road [12]. However, it has since become apparent that pure behavioral cloning suffers from distribution shift--a phenomenon whereby the distribution of states visited by the policy diverges from that of the original training data set by way of incremental error, eventually leading to poor predictions by the model and irrecoverable deviation from the intended task. To improve the ability of policies to recover from this failure mode, various more complex approaches to the task have been proposed. One major direction of research has been the use of inherently robust, composable functions known as _dynamic motion primitives_ (DMPs), employing systems of differential equations and parametric models such as Gaussian basis functions to obviate the problem of distribution shift entirely--ensuring convergence toward the goal through the explicit dynamics of the system [13]. Though showing promising results, it must be noted that the model templates used are quite domain specific. Others have attempted to use more general-purpose models in conjunction with interactive sampling of the expert response to compensate for distribution shift [14]. While theoretically able to guarantee convergence, the major drawback of such methods is that an expert function is required that can be queried numerous times as part of the training process. This severely restricts applicability to practical use cases since it is impossible to implement when only given a set of pre-recorded demonstrations. A different means of handling this problem is given by the field of _inverse reinforcement learning_[4]. Rather than attempting to model the expert function directly, it is assumed that the expert is itself acting to maximize an unknown reward function. An attempt, therefore, is made to approximate this reward in a way that explains the observed behavior. From there, this becomes a classical reinforcement learning problem, and any training method or model architecture developed in the space of this adjacent field of study can be employed. Where early approaches made certain assumptions about the form of this reward function--such as it being linear--more recent work has proposed that generative adversarial networks be used where the discriminator can approximate the class of all possible reward functions fitting the observations over an iterative training process [15; 16]. Perhaps the most promising results, however, come from the sequence modeling domain. Recurrent neural networks show up in earlier scientific literature periodically, such as in predicting a time-series of robot end-effector loads in an assembly task [17] and learning latent action plans from large, uncategorized play data sets [18]. However, current state-of-the-art performance across a wide variety of sequence prediction tasks--among them being imitation learning in a robotics context--is given by combining a large, universal transformer model with embedding schemes specific to various data modalities [19]. These results strongly suggest that structuring one's approach to be compatible with general-purpose sequence predictor algorithms is preferable for ensuring its longevity. When it comes to the use of motion capture data, previous work in the robotics and imitation learning corner is quite sparse, possibly due to the costly and specialized nature of the equipment involved. One previously considered direction is the use of consumer-grade motion tracking equipment for collecting demonstrations [20].