# Unsupervised Offline Changepoint Detection Ensembles
## 4 Discussion  
We provide here our recommendations based on the experiment results.  
* According to the experiment results, the ensemble approach almost always is a priority option. Among non-ensemble procedures, the favorite ones are Opt and BinSeg methods but only with _mahalanobis_ cost function. Among ensemble search algorithms, we recommend to use OptEnsemble.
* The question of what combination function to select is mentioned in [13]. Averaging and maximum aggregation functions are the only methods compared by authors from a bias-variance trade-off perspective. None of these functions were declared as a clear winner. The experiments indicated that maximization achieved better results than averaging for larger datasets, while the opposite picture was for smaller datasets. Our experiment revealed that selecting either \(Sum\) or \(WeightedSum\) is a confident strategy for steadily achieving high scores. \(Min\) aggregation function may lead as to the best or the worst score, while it still can be used in combination with OptEnsemble and \(MinMax\) or \(Rank\) functions without major losses in a score. After all, we either cannot highlight the best combination strategy.
* Regarding the scaling functions, we can recommend avoiding the \(Rank\) to maximize the results for most cases, even though it sometimes allows a high score. All of the other scaling functions get similar results.  
The limitations of the study are mainly connected with the data properties and generalizability of the results. Our research is mainly focused on the technical domain and industrial data, hence industrial faults and failures. Not only do the datasets used in our numerical experiment not allow generalizing results to any domain of knowledge, but the bias in the data in some specific domain may lead to the inconsistency of the results. Differences in the real-world data are forcing researchers to test the methods when expanding the proposed ensembles to the new application. One more limitation is connected with the NAB scoring algorithm used for the evaluation of various methods results. The way of  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Place** & **Algorithm** & **Standard** & **LowFP** & **LowFN** \\ \hline \multicolumn{5}{c}{TEP} \\ \hline
1 & OptEnsemble & 41.81 & 41 & 42.16 \\
2 & BinSegEnsemble & 41.81 & 41 & 42.16 \\
3 & Opt & 36.88 & 35.82 & 37.29 \\
4 & BinSeg & 36.88 & 35.82 & 37.29 \\ \hline - & LSTM [47] & 37.3 & - & - \\ - & DPCA [47] & 8.6 & - & - \\ \hline \multicolumn{5}{c}{SKAB} \\ \hline
1 & MSCRED [48] & 28.74 & 23.43 & 31.21 \\
2 & LSTM [49] & 27.09 & 11.06 & 32.68 \\
3 & BinSeg & 24.1 & 21.69 & 25.04 \\
4 & OptEnsemble & 23.07 & 20.52 & 24.35 \\
5 & Opt & 22.37 & 19.9 & 23.37 \\
6 & Hotelling’s \(T^{2}\)[50] & 17.87 & 3.44 & 23.2 \\
7 & ARIMAFD [46] & 16.06 & 14.03 & 17.12 \\ \hline \end{tabular}
\end{table}
Table 4: Comparison of the CPD and CPDE procedures with SOTA changepoint detection algorithms.  
applying the NAB algorithm may differ from varying the window size and location around the changepoint to application profile tunning, and no recommendations or algorithm limitations are provided. The major limitation of the work is that the CPDE approach was not studied for the unknown number of changepoints. The last limitation is connected with some algorithm parameters selected from the grid search procedures. They are the number of lags for the autoregressive model equal to 1 and the window size equal to 20 points for the Win and WinEnsemble search methods. We assumed that the grid search procedure provides optimal results for these parameters. Nevertheless, the limitations were known during the study's design, and we consider that they did not affect the principal results of the numerical experiment.