## 7 Results and Analysis  
### Anomaly Detection in Kitting Experiment  
According to Section 4, the anomaly detector implementation consists of representing multimodal observations through latent states, concatenating the derived log-likelihood values of a latent state, and calculating the anomaly detection threshold.  
We first show a visualization of the latent state sequences to illustrate the latent state partitions for nominal demonstrations. Figure 7 illustrates hidden-state partitions for 20 nominal executions for Skill 3 that were carried out with varying speeds and goals. Notice how the non-parametric method yields varying dynamics across executions reflecting statistical variations in the robot motion trajectory due to changes in the desired trajectory completion time and varying goal distances.  
Then, latent state log-likelihood values are concatenated, and the corresponding thresholds computed. To this end, we visualized the performance of the proposed method as well as a baseline by assessing the anomaly time reaction of anomaly flags; that is, which algorithm can make a correct detection at a time closer to the ground truth. For training, ten nominal executions were used along with 3-fold cross validation to obtain nominal sHDP-VAR-HMM models for each skill. The other 20 nominal executions were used for testing.  
Figure 7: Illustration of the hidden-state representation of Skill 3 when the truncation states number is set to \(K=5\). Each row represents the learned hidden-state sequence of a nominal trial. Each color represents a different hidden state. White color indicates a shorter skill duration.  
The execution varying threshold for anomaly detection was set according to Equation (29). The results for Skill 3 are visualized in Figure 8. As seen in Figures 7 and 8, latent state \(zhat=2\) (the orange state in Figure 7) is the most prominent latent state (the majority of the observations in the skill are ascribed to this state). We now analyze the time sensitivity of our system in detecting anomalies. Anomaly identification sensitivity is intimately connected to threshold setting. We compare the results of our execution varying threshold with a fixed threshold presented in [2]. The comparison is seen in Figure 9. Under nominal situations, both methods have the same robustness in avoiding false positives. In anomalous situations, there are multiple occasions in which our varying threshold yielded better time sensitivity and detection accuracy. From Figure 9, we see that for executions 2 and 4 our varying threshold detector identified anomalies that the fixed threshold method could not. Notice that the distance between the _anomaly_t_by_human_ and the _anomaly_t_by_detector_ is larger for the fixed threshold (recall from Section 6.2 that ground truth time stamps were obtained during training). This improved sensitivity reduces false negatives and helps us flag anomalies at times closer to the ground truth.  
Finally, Table 1 shows the anomaly detection performance for our hidden-state detector (HSD) and the baseline gradient-based detector (GD) for the six skills used in the kitting task for 368 nominal executions and 136 anomalous executions. Our results indicate that while the GD system had the HSD resulting in 16.6% better precision, our system had 61.7% better recall. The difference is significant in reducing false negatives and significantly raising the sensitivity of our system. The sensitivity improvement is almost four times bigger than the difference in precision. For this reason, our system reflects an F1 score that is 25.0% larger and an accuracy that is _F1 score_, and an accuracy that is 5.7% higher, standing at 91.0%.  
Figure 8: All the log-likelihood values of individual hidden state of Skill 3 when the truncation states number is set to \(K=5\). Each subplot shows the mean value (black dotted line) and the corresponding log-likelihood threshold value (red solid line) in Equation (29).