# Semi-Supervised Time Series Anomaly Detection Based on Statistics and Deep Learning
## 2 Background Knowledge  
### Time Series and Related Models  
A time series \(X=(x_{1},x_{2},\dots)\) is a sequence of observations (or data) ordered chronologically, where \(x_{t}\) is recorded at a specific time point \(t\), where \(t\) is a positive integer [47]. It is usually assumed that time points are equally separated. Based on the properties of observations, we have two types of time series. On the one hand, if observations of a time series are scalar values, then the time series is a univariate time series. On the other hand, if observations are multi-dimensional vectors, then the time series is a multivariate time series. This paper focuses on univariate time series; therefore, we use \"time series\" to stand for \"univariate time series\" in the following context. Time series can be either stationary or non-stationary. According to [47], the definition of a stationary time series is given below. A time series \(X\) is strictly stationary if the probability distribution of \(X\) does not change with time shifts. In other words, for every \(m\) and \(n\), the distributions of \(Y=(x_{1},\dots,x_{n})\) and \(Z=(x_{1+m},\dots,x_{n+m})\) are the same, where \(Y\) and \(Z\) are subsequence of \(X\). A time series \(X\) is weakly stationary if its mean \(E(\cdot)\), variance \(Var(\cdot)\), and covariance \(Cov(\cdot)\) do not change by time shifts. In other words, for every \(m\) and \(n\), \(E(Y)=E(Z)=\) a constant \(\mu\), \(Var(Y)=Var(Z)=\) a constant \(\sigma^{2}\), and \(Cov(Y,Z)=\gamma(1\,m-n\,1)\) for some function \(\gamma(\cdot)\). In this paper, a time series is assumed to be stationary if it is weakly stationary. The next subsection will show how to use the Dickey-Fuller test to check the stationarity of a time series. Stationary time series can be easily formulated by the auto-regressive (AR) [48], moving average (MA) [48], and auto-regressive and moving average (ARMA) [48] models, whereas non-stationary time series can be properly formulated or processed by the auto-regressive integrated moving average (ARIMA) [48] model and the seasonal auto-regressive integrated moving average (SARIMA) [41] model, and the seasonal-trend decomposition based on LOESS (STL) method [42]. Below, the models and the method are described one by one. Let \(X=(x_{1},x_{2},\dots)\) be a stationary time series and \(x_{t}\) in \(X\) be an observation recorded at a specific time point \(t\), where \(t\) is a positive integer. In consideration of \(p\) earlier lags \(x_{t-1}\), \(x_{t-2}\), \(\dots\), \(x_{t-p}\) and the white noise (or random error) \(e_{t}\) at time \(t\) with the zero mean and a bounded variance, \(x_{t}\) can be formulated according to the auto-regressive AR(\(p\)) model, as shown in (1). \[x_{t}=a_{1}\ x_{t-1}+a_{2}\ x_{t-2}+\dots+a_{p}\ x_{t-p}+e_{t}, \tag{1}\]where \(p\) is the order and \(a_{1},a_{2},\ldots,a_{p}\) are coefficients or parameters of the AR model. In consideration of the random error \(e_{t}\) at time \(t\) and \(q\) previous random errors \(e_{t-1}\), \(e_{t-2},\ldots,e_{t-q}\), the deviation of \(x_{t}\) from the mean \(\mu\) of the whole series can be formulated according to the moving-average MA(\(q\)) model, as shown in (2). \[x_{t}-\mu=e_{t}+b_{1}\,e_{t-1}+b_{2}\,e_{t-2}+\ldots+b_{q}\,e_{t-q}, \tag{2}\]  
where \(q\) is the order and \(b_{1},b_{2},\ldots,b_{q}\) are coefficients or parameters of the MA model. In consideration of \(p\) earlier lags \(x_{t-1}\), \(x_{t-2},\ldots,\)\(x_{t-p}\), \(q\) pervious random errors \(e_{t-1}\), \(e_{t-2},\ldots,e_{t-q}\), and the random error \(e_{t}\), \(x_{t}\) can be formulated according to the auto-regressive and moving-average ARMA(\(p\), \(q\)) model, as shown in (3). \[x_{t}-(a_{1}\,x_{t-1}+a_{2}\,x_{t-2}+\ldots\,+a_{p}\,x_{t-p})=e_{t}+b_{1}\,e_{ t-1}+b_{2}\,e_{t-2}+\ldots\,+b_{q}\,e_{t-q}, \tag{3}\]  
where \(p\) is the order and \(a_{1},a_{2},\ldots,a_{p}\) are coefficients of the AR model, and \(q\) is the order and \(b_{1}\), \(b_{2},\ldots,\)\(b_{q}\) are coefficients of the MA model.