# SMM: Leveraging Metadata for Contextually Salient Multi-Variate Motif Discovery
## 5 Experimental Evaluation
### Motifs Used as Ground Truth  
For a fair evaluation, as is common in the literature, we inject true motifs onto time series generated through random walks. Then, the injected motifs are used as labeled ground truth (Ground truth data are available at [[https://drive.google.com/drive/folders/1](https://drive.google.com/drive/folders/1) NBvywswb1 wzryIwhIW2 xiS17 ns_xF03J?usp=sharing]([https://drive.google.com/drive/folders/1](https://drive.google.com/drive/folders/1) NBvywswb1 wzryIwhIW2 xiS17 ns_xF03J?usp=sharing), accessed on 20 May 2021). In order to ensure that the simulated series are meaningful representations of the original time series, we first compute the average displacement for the series and, at each step of the random walk, we make a move bounded by a percentage of the original average displacement. In the experiments, we considered 10, 25, 50, 75 and 100% average displacement. In addition, we considered a (0-1) random walk where the average displacement was bounded simply by 1 unit (this corresponds to a mostly idle motion pattern). The ground truth motifs are created and injected as follows:  
1. First, we identify and extract multi-variate dominant behaviors/patterns in the form of RMT features (robust multivariate time-series feature) from the series in a given dataset. 2. Then, we select a random set of distinct patterns to be injected as motifs. 3. Next, we generate a set of multi-variate random walks to be used as the motif-free time series; to ensure that the generated series closely match the characteristics of the original series, we perform the following operations: * We compute, from each motif pattern, the value of average displacement; * We scale the generated random walks to match a specific percentage of the _average displacement_ of the motif patterns for making the motif instances injected consistent within the global behavior over the time series. 4. We finally inject the selected patterns onto the created motif-free random walks. Figure 8: Heatmap visualization of sample time series: each row corresponds to a different variate (best viewed in color). (**a**) MOCAP sample. (**b**) Building energy sample. (**c**) BirdSong sample. We record, as ground truth, the variates on which the patterns have been injected along with the temporal scopes of the pattern insertions. Table 2 provides an outline of the characteristics of the time series that are generated through the above process:  
* _Number of motifs (\(m\))_: this is the number of motifs inserted into a single multivariate series;
* _Recurrence count (\(k\))_: this is the number of patterns inserted for each motif;
* _Length flexibility (\(f\))_: for a given motif, we allowed the length of its subsequences to vary--\(f\) denotes the lowerbound on the length of the subsequences as a percentage of the original pattern. In addition to the above, we considered alternative variate alignment and placement strategies for the injected motifs; motifs are inserted either on the original variates (\(v\)-_aligned_; default), or on variates that are structurally similar to the original ones (\(v\)-_structure_). In the experiments reported in this section, we compare the proposed SMM algorithm to one of the most recent multi-variate motif proposals, the matrix-profile strategy [25], which we refer to, in this section, as matrix profile with clustering (MPC). Note that the original matrix-profile algorithm identifies recurring pairs of multi-variate subsequences; given all the recurring sub-sequences extracted from the series, we apply an adaptive k-medoids strategy (similar to the one applied for SMM, Section 4.2, but using the distance measure presented in [25] instead of descriptor distances) on these subsequences to obtain the corresponding frequently recurring patterns. The MPC algorithm takes, as input, a target window size (set to 60 for Energy and MoCap, 32 for BirdSong), corresponding to the default length of patterns inserted (considering each possible dataset) into the series, as described above.