When Complexity Is Good: Do We Need Recurrent Deep Learning For Time Series Outlier Detection?

1 INTRODUCTION . Outlier detection is the process of identifying unusual data points , or collections of data points within a dataset . Within different contexts , these outliers can take various forms , for example : • Credit card fraud detection : Unusual spending . • Internet traffic monitoring : Unusual surges in traffic or security monitoring . • Dementia care : Unusual patterns of activity , unusual sleep/wake patterns at night . This paper will focus specifically on time series outliers . These are outliers that appear in sequential data , which Blázquez-Garcı́a et al . ( 2021 ) argued could take three main forms : • Point outliers : These are single data points in time that are unusual from the rest of the dataset when considering other values in the time series or points locally . Point outliers could have global unusual values , or contextually ( in our case , temporally ) unusual values . • Subsequence/Collective outliers : These are consecutive points whose behaviour as a collective is unusual , but not necessarily if considered separately . • Outlier time series : These are instances in which an entire time series ( for example , a variable in a multivariate dataset ) can be considered unusual . In this paper , we focus on point and subsequence outliers and discuss when and how recurrent deep learning techniques could be used for this type of outliers compared to non-recurrent techniques . This involves testing several algorithms on multiple datasets containing different outlier types . We also introduce two new outlier detection methods by modifying the Long Short Term Memory Recurrent Neural Network ( LSTM-RNN ) ( Hochreiter & Schmidhuber , 1997 ) outlier detection technique presented in Bontemps et al . ( 2016 ) ; Singh ( 2017 ) . We change the underlying forecasting model and instead of an LSTM-RNN , we train a transformer model ( Vaswani et al. , 2017 ) and similarly , use its prediction loss as the basis for the outlier score calculation . We also removed the focus on collective outliers ( as is in the LSTM-RNN outlier detection model ) , so that our model was applicable to both point and collective outliers . We continue this paper by introducing some of the preliminary concepts that will be required to design the outlier detection settings , as well as explaining the difference between recurrent and non-recurrent based outlier detection algorithms . Next , this paper discusses the related works and describes how our study extends the existing work . We introduce the methods and models that will be used to answer the questions of this paper , followed by the results and discussion on their significance . This paper ’ s aim is to answer the following question : “ When detecting outliers in time series data , is it appropriate to use recurrent neural networks and how do we design models for this purpose ? ” Our motivation for answering this question is to enable a better understanding of which outlier detection techniques would be most usefully applied to our healthcare monitoring ” real-world ” time series dataset , containing movement information collected form the homes of people living with dementia ( more information on Section 2.1 ) . We also wish to understand how useful recurrent based algorithms are when points in the dataset contain complex temporal relationships with each other . For example , we wish to find outliers that are not only locally and globally unusual in their values , but where they are also unusual given some sequence that they exist in . 1.1 PRELIMINARY CONCEPTS . WHAT IS A RECURRENT NEURAL NETWORK ? Within this paper , we will refer to two different types of outlier detection methods . Those that use non-recurrent techniques and those that rely on recurrent deep learning models . Recurrent neural network approaches are characterised by their use of sequences in the time series data to detect outliers . Classical non-recurrent algorithms assume no temporal relationship between data points and calculate outlier scores on a point-by-point basis , whilst recurrent neural network algorithms use temporal relationships when detecting outliers . In theory , this difference should make recurrent neural network models superior to non-recurrent models when detecting outliers in complicated time series data . This is not necessarily the case however , since many outliers will hold unusual enough values such that they can be detected by non-recurrent techniques . In addition , extra capacity for learning complicated relationships in the data brings larger numbers of hyperparameters to tune , which can cause difficulty in training robust models . This hyper-parameter tuning becomes an issue when applying complex techniques to new datasets for which there is no outlier ground truth to measure model performance . OUTLIERS IN A DATASET As discussed in Section 1 , outliers take three main forms when viewed in the context of time series data . The number of outliers in a dataset is defined as a proportion of the whole dataset and is called the contamination . To contaminate a dataset is to add outliers to that dataset . When measuring the performance of outlier detection techniques , we will use a harmonic mean of precision and recall , defined as the F1 score ( see Appendix A.1 ) . 1.2 RELATED WORK . There has been a long history of developing outlier detection methods and exploring different algorithms for time series outlier detection ( Lai et al. , 2021b ; Blázquez-Garcı́a et al. , 2021 ) . The existing works propose various methods to identify different types of outliers that are present in time series datasets . Lai et al . ( 2021b ) show comparisons between different algorithms for outlier detection and evaluate their performance on both synthetic and real-world datasets . However , Lai et al . ( 2021b ) do not directly compare recurrent and non-recurrent based outlier detection algorithms . Elsayed et al . ( 2021 ) aimed to address a similar question in relation to time series forecasting , in which they argued that a carefully configured Gradient Boosted Regression Trees ( GBRT ) model ( Friedman , 2001 ) could compete and even sometimes out-perform many complex deep learning models for time series forecasting . Pang et al . ( 2021 ) studied the role of deep learning in outlier detection , in which they acknowledge that complex models would be useful for complex anomalies ( those with complicated spatial or temporal dependencies ) , although it is not as widely explored as evaluating the performance of complex models for detecting point anomalies . Both of these studies , however , do not directly compare recurrent and non-recurrent based algorithms . We aim to extend the scope of Lai et al . ( 2021b ) ’ s work by specifically investigating whether recurrent neural networks are useful for outlier detection compared to non-recurrent methods . An answer to this question would guide us in our search for a reliable outlier detection method for application on our remote monitoring time series dataset , related to dementia care ( described in Section 2.1 ) . We are aware that our dataset will contain complex temporal relationships between data points and we wish for an outlier detection algorithm to be proficient at finding all of these types of outliers . 2 METHODS AND MODELS . In this section , we discuss different datasets used in our experiments and describe how the datasets are generated or collected . Next , we discuss which existing recurrent and non-recurrent outlier detection models we have chosen . We also introduce two new outlier detection methods that rely on transformer models ( Vaswani et al. , 2017 ) . 2.1 DATASET . To evaluate the models , we have used the TODS python package 1 and benchmarks ( Lai et al. , 2021a ) , which contain several synthetic and real-world datasets for time series outlier detection . The synthetic datasets were generated using sinusoidal waves with various frequencies and amplitudes . These datasets are multivariate and always consist of 5 variables . Contamination functions are used to contaminate this dataset with 5 different objectives as outlined in Lai et al . ( 2021b ) . These contamination objectives are aimed at producing point outliers or collective outliers . We test the algorithms in this paper on 15 different datasets , with different combinations of types of contamination . A more thorough explanation of the way that these datasets are generated is given in Lai et al . ( 2021b ) . The TODS package also readily makes available two real-world datasets ( see Appendix A.2.1 for more information ) , labelled with outliers relating to the context . Those two datasets are : • SWAN-SF : SWAN-SF was collected by the Harvard Dataverse and contains space weather data ( Angryk et al. , 2020a ) . This dataset has a contamination factor of 23.8 % . • GECCO : This is a dataset collected by SPOTSeven Lab 2 for a data challenge in 2018 and contains water quality information . This dataset has a contamination factor of 1.2 % . These datasets were processed by Lai et al . ( 2021b ) before being made available for outlier detection . The code that processes the data is readily available in the TODS package ( see Section 6 for information on reproducibility ) . Alongside these datasets , we also tested different models on our dementia care dataset ( referred to in this paper as the Movement dataset ) . This data contains movement information , collected from over 140 people living with dementia in their real home settings for a period of over 2 years ( from February 2019 to June 2021 ) . Each feature in this dataset represents the number of times a sensor was triggered in a 3 hour period in a location of the home . To simplify this , we will think of the sensor triggers as a visit to a room containing the sensor . The dataset contains 5 of these locations , which represent the hallway , living room , bedroom , bathroom and kitchen . These 3 hour time windows were then concatenated to form points that cover 24 hour periods and contain 40 features . This means that each of these features represents a frequency of visits to a location in a particular 3 hour window . The outliers were labelled as points in the data in which the person living with dementia was admitted to hospital or had an infection . The labelling is done by a clinical 1https : //github.com/datamllab/tods/tree/benchmark 2https : //www.spotseven.de/ monitoring team who have been part of the study and interact with the participants based on the observations and measurements collected by the in-home monitoring technologies integrated into a clinical dashboard . Only the people living with dementia who had at least one labelled outlier , were included in the final dataset . The data was labelled as an outlier 4 days preceding and proceeding the infection and hospital admission episodes . Data points in which all of the feature values were 0 and there was no hospital admission or infection were then removed . This leaves the contamination factor of the dataset at 2.3 % . Graphs showing the distributions of values in the dataset can be viewed in Appendix A.2.2 . After applying the methods to the dataset , we split the data by each source ( a household containing a person living with dementia ) and tested the algorithms on each of these . We removed any sources in which there were less than 5 outliers labelled . We will refer to this dataset as the separated Movement data . The datasets chosen span a wide range of contamination values , allowing the algorithms to be evaluated over a range of scenarios . They also contain a mix of ratios of point outliers to collective outliers . See Appendix A.2 for more information . We assumed that all of the datasets either have point outliers or collective outliers . Point outliers are defined by a single outlier on its own , whilst collective outliers are defined by consecutive outlier points in a time window .

Summary: This paper studies time series outlier detection. The authors propose to compare recurrent and static outlier detection methodologies on multiple synthetic and real-world temporal datasets. Additionally, they present an extension to the LSTM outlier detection using an attention-based approach. The analysis suggests that non-recurrent methodologies should be prefered as they perform marginally better while requiring fewer parameters and training. Only in the context of complex temporal relations, recurrent approaches should be used.