# Scale-BOSS-MR: Scalable Time Series Classification Using Multiple Symbolic Representations +
## 4 Evaluation  
Aiming to balance between accuracy of predictions and efficiency in training and testing, we evaluate instantiations of the SCALE-BOSS-MR framework in comparison to the state-of-the-art methods, in terms of the mean total time and mean accuracy.  
The mean total time is the total execution time (both train and test time) for all datasets divided by the total number of datasets. The mean accuracy is the average accuracy across all datasets. We also provide the standard deviation (std) of the total execution time and of accuracy.  
Accuracy and execution time are the commonly used measures to report performance in most works in the literature, e.g., [11]. This choice allows us to be directly comparable to other algorithms. However, to comprehensively present the strengths and limitations of SCALE-BOSS-MR, we directly compare SCALE-BOSS-MR configurations with state-of-the-art algorithms such as WEASEL 2.0, Rocket, MiniRocket, and MRSQM.  
While Appendix A reports on accuracy and total execution time for individual time series datasets, we report on mean accuracy and mean total execution time in the main part of the paper, comprehensively providing the strengths and limitations of the proposed framework.  
Table 1 shows the characteristics of the UCR datasets used in the evaluation. UCR time-series datasets are the de facto datasets for time series classification, and we have chosen to use the eight datasets from the UCR time series repository with the largest training set. _Train_size_ denote the number of time series in the training set, _test_size_ denote the number of time series in the test set, _n_classes_ denotes the number of classes in the dataset, and _n_timestamps_ denotes the length of each time series in the dataset. The datasets are split into train and test time series by the dataset providers.  
Table 2 specifies the configurations of window sizes and window steps. When the value is a float, the float represent the percentage of the size of time series. If the value is an integer, then this denotes the number of subsequent time series points included in the window. Integer values of the window sizes are those used by WEASEL in [18]; however, with sufficiently large differences between subsequent window configuration sizes (i.e., four). Similarly, float values are those provided by ptyts as initial values, and we add to them to these further configurations for exploration.  
\begin{table}
\begin{tabular}{l r r r r} \hline \hline
**Name** & **Train\_Size** & **Test\_Size** & **n\_Classes** & **n\_Timestamps** \\ \hline Crop & 7200 & 16,800 & 24 & 46 \\ FordB & 3636 & 810 & 2 & 500 \\ FordA & 3601 & 1320 & 2 & 500 \\ NonInvasiveFetalECGThorax2 & 1800 & 1965 & 42 & 750 \\ NonInvasiveFetalECGThorax1 & 1800 & 1965 & 42 & 750 \\ PhalangesOutlinesCorrect & 1800 & 858 & 2 & 80 \\ HandOutlines & 1000 & 370 & 2 & 2709 \\ TwoPatterns & 1000 & 4000 & 4 & 128 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Characteristics of the datasets.  
Table 3 shows dilation configurations in terms of the dilation factors. These dilation configurations are those mostly suggested in [11]. We subsequently perform a thorough evaluation to determine the best dilation configuration parameters in order to balance between accuracy and execution time.  
To present the results, we use the following SCALE-BOSS-MR configurations' naming convention:  
NAME-CLS_METHOD-WINDOW_CONFIG-USE_OF_TREND-DILATION_CONFIG-NGRAM,  
where  
* [noitemsep=-1 ex,  
Subsequently, we describe the major findings from the results reported in these tables and provide further results that allow us to show the potential of the proposed approach.