# A Motion Capture and Imitation Learning Based Approach to Robot Control
## 5 Implementation
### Pre-Processing, Extraction of Implicit Control Signals  
Figure 3 provides a schematic overview of the main steps involved in the data preparation process. For illustrative purposes, graphs of the effector \(x\)-coordinate with respect to time are used, but the final picture also contains the estimated target coordinate of each throw. After each recording session, an entirely unstructured corpus is available. It contains timestamped pose observations collected at whatever intervals they happened to have been generated by the tracking software. Each observation only contains information about a single tracked body. A peculiarity of this particular recording method is that observations about each body tend to arrive in bursts:  
\[\mathbf{s}_{t1}^{a},\mathbf{s}_{t2}^{b},...,\mathbf{s}_{tm-1}^{b},\mathbf{s}_ {tm}^{b} \tag{7}\]  
where \(\mathbf{s}_{t\in\mathbb{R}}^{a}\) corresponds to the observed state of object \(a\) at time \(t\). Thus, to obtain sequential data in discrete time, it is necessary to resample the positions and orientations of all relevant bodies at a constant time step:  
\[(\mathbf{s}_{1}^{a},\mathbf{s}_{1}^{b},\mathbf{s}_{1}^{c}),...,(\mathbf{s}_{ k}^{a},\mathbf{s}_{k}^{b},\mathbf{s}_{k}^{c})=\mathbf{s}_{1},...,\mathbf{s}_{k} \tag{8}\]  
Figure 2: Motion capture equipment and demonstration acquisition process. A sampling frequency of 100 Hz was selected to obtain spatial precision on the order of centimeters given the velocities involved. Fortuitously, the intervals between observed bursts in the actual data are generally on this order as well. Resampling is accomplished using forward fill interpolation, which introduces a slight sawtooth oscillation, but this is removed in the subsequent smoothing step along with other discontinuities. After resampling, individual demonstrations are separated using the aforementioned consistent starting position. This may vary from recording session to recording session, so it needs to be identified and configured manually. Specifically, the condition used to identify demonstration start points is as follows:  
\[t_{start}\in[t]_{1}^{k}:t<t_{start}\wedge(t_{start}-t<\Delta t_{ max})\Rightarrow\] \[\Rightarrow x_{t}\in(x_{0}-\delta x,x_{0}+\delta x),y_{t}\in(y_{0}-\delta y,y_{0}+\delta y) \tag{9}\]  
which is to say that every observation no more than \(\Delta t\) steps before \(t_{start}\) has to lie within \(\pm\delta x,\pm\delta y\) of the datum \((x_{0},y_{0})\). Each demonstration then consists of observations  
\[(\mathbf{s}_{start},...,\mathbf{s}_{start+steps}),steps\in\mathbb{N} \tag{10}\]  
The constants \(\Delta t,\delta x,\delta y,steps\) are determined experimentally to produce consistent observations for the particular task and may require tuning if the manner in which demonstrations are recorded changes. The newly split demonstrations are saved in separate files, thus allowing demonstrations generated in different sessions to be processed at once. As further thresholding steps rely on estimates of pose derivatives, a smoothing step is first employed to remove noise and the slight sawtooth oscillation induced by the resampling step. This is accomplished by applying a rolling average kernel to the trajectory. Nevertheless, it was found that any estimate of derivatives would exacerbate what noise there was in the dataset, leading to inconsistent results. So a simplified estimator  
Figure 3: Pre-processing pipeline overview. All charts are purely illustrative and show effector \(x\)-axis position with respect to time (in blue). The final diagram also shows the target \(x\)-coordinate of each discrete demonstration (in red). The thresholds are, from left to right: movement, release, freefall start and freefall end.