# Semi-Supervised Time Series Anomaly Detection Based on Statistics and Deep Learning
## 2 Background Knowledge
### Time Series and Related Models
By introducing the lag operator \(L\), where \(Lx_{t}=x_{t-1}\), and \(L(Lx_{t})=L^{2}x_{t}=x_{t-2}\), \(\ldots,\) and so on, we can rewrite ARMA(\(p\), \(q\)) according to (4). \[(1-\sum_{i=1}^{p}a_{i}L^{i}\,)\,x_{t}=(1+\sum_{i=1}^{q}b_{i}L^{i}\,)e_{t} \tag{4}\]  
Let \(X=(x_{1},\,x_{2},\ldots,\)) be a non-stationary time series and \(x_{t}\) in \(X\) be an observation recorded at a specific time point \(t\), where \(t\) is a positive integer. As \(X\) is not stationary, it cannot be formulated by AR, MA, or ARMA models. However, [PERSON] and [PERSON] [49] suggested that differencing a non-stationary time series one or more times can make it stationary. To be specific, by differencing the non-stationary time series \(X\) for \(d\) times, \(X\) can still be properly formulated by the ARIMA(\(p\), \(d\), \(q\)) model, as described below. Note that \((1-L)x_{t}=x_{t}-x_{t-1}=\) first-order difference, and \((1-L)^{d}\,x_{t}=d^{th}\)-order difference. We have the following Formula (5). \[(1-\sum_{i=1}^{p}a_{i}L^{i}\,)(1-L)^{d}x_{t}=(1+\sum_{i=1}^{q}b_{i}L^{i}\,)e_ {t} \tag{5}\]  
The Box-Jenkins approach leads to the ARIMA model to formulate a non-stationary time series. It has three steps [48], as elaborated below. Step 1: Determining the order \(d\) of differencing to make the time series stationary:  
This step can be achieved by repeatedly differencing the time series and by applying the Dickey-Fuller test to check whether or not the time series has become stationary. The Dickey-Fuller test will be descried later in the next subsection. Step 2: Identifying the model:  
This step can be achieved by applying the auto correlation function (ACF) and the partial auto correlation function (PACF) to determine which model is most suitable to formulate the time series. Possible models are AR(\(p\), \(d\)), MA(\(d\), \(q\)), and ARMA(\(p\), \(d\), \(q\)). Step 3: Estimating the parameters:  
No matter which of AR(\(p\), \(d\)), MA(\(d\), \(q\)), and ARMA(\(p\), \(d\), \(q\)) is determined in the previous step, it is a linear regression model. Therefore, this step can be achieved by applying optimization mechanisms, such as the maximum likelihood estimator, to find model coefficients just like finding linear regression coefficients [50]. As just mentioned, the ARIMA model formulates the time series trend with three trend parameters: the auto-regression order \(p\), the difference order \(d\), and the moving average order \(q\). By adding four seasonal parameters, the SARIMA (seasonal auto-regressive integrated moving average or seasonal ARIMA) model further considers the seasonal component by adding four seasonal parameters: the seasonal auto-regressive order \(P\), the seasonal difference order \(D\), the seasonal moving average order \(Q\), and the number \(m\) of time points for a single seasonal period. A SARIMA model is specified as SARIMA(\(p,d\), \(q\))(\(P\), \(D\), \(Q\))\({}_{m}\). In addition to \(d\)-order differencing of ARIMA, seasonal ARIMA or SARIMA also includes the \(m\)-order seasonal differencing to reduce the seasonal component. Please refer to [41] for the details of adding seasonal differencing terms for the SARIMA model. Below, the STL method [42] is introduced, where STL stands for \"seasonal-trend decomposition based on LOESS\" and LOESS stands for \"locally estimated scatterplot smoothing\". The STL method decomposes a time series into three components, the trend, seasonality, and remainder. A trend shows a persistent increasing or decreasing tendency in data, seasonality indicates seasonal effects over a fixed period of time, and the remainder corresponds to the data noise. Specifically, the STL method runs iteration by iteration to decompose time series by finding the data trend and seasonal data trend with the help of the smoother LOESS [51]. The basic idea of LOESS is that a function can be well fitted in a small neighborhood by a low-order polynomial. LOESS runs fast and fits data properly, which in turn makes STL run fast and well.