# A Survey on Change Detection and Time Series Analysis with Applications
## 2 Decomposition Methods into Frequency Domain
### Least-Squares Spectral Analysis (LSSA)  
The LSSA is an appropriate method of analyzing unequally spaced time series that comprise trends and/or datum shifts [10; 11]. The LSSA and its modifications have been described in detail in many publications [3; 12; 13; 14]. Its basic definition for time series is described below. Suppose that \(\mathbf{f}\) is a real times series of \(n\) data points, \(\mathbf{f}=\left[f(t_{1}),f(t_{2}),\ldots,f(t_{n})\right]^{\mathrm{T}}\), where \(\mathrm{T}\) is the transpose and \(t_{j}\) may not be equally spaced. Furthermore, suppose that \(\mathbf{C_{f}}\) is the covariance matrix of dimension \(n\) associated with the time series. Note that each data point \(f(t_{j})\) can be generally assumed to be a random variable with mean \(\mu_{j}\) and variance \(\sigma_{j}^{2}\), so the covariance between any two data points may be calculated [6] (Chapter 3). Let \(\mathbf{P}=\mathbf{C_{f}^{-1}}\), and \(\Omega\) be a set of cyclic frequencies (positive real numbers) that may be chosen based on the scope of analysis. The LSSA estimates a vector \(\mathbf{c}=[c_{1},\ldots,c_{m}]^{\mathrm{T}}\) satisfying \(\mathbf{f}=\mathbf{\Phi c}\). Matrix \(\mathbf{\Phi}\) of dimension \(n\times m\), which models the physical relationship between the time series (the vector of measurements) and the unknown vector \(\mathbf{c}\), is called the design matrix [10]. If the dimension of \(\mathbf{f}\) is larger than \(m\), then the system \(\mathbf{f}=\mathbf{\Phi c}\) is overdetermined, i.e., the system has more equations than unknowns. In order to solve the system, one can recall the least-squares principle to minimize quadratic form \((\mathbf{f}-\mathbf{\Phi c})^{\mathrm{T}}\mathbf{P}(\mathbf{f}-\mathbf{\Phi c})\). This results in the well-known expression  
\[\mathbf{\hat{c}}=(\mathbf{\Phi}^{\mathrm{T}}\mathbf{P}\mathbf{\Phi})^{-1} \mathbf{\Phi}^{\mathrm{T}}\mathbf{Pf}, \tag{5}\]  
where \(\mathbf{\hat{c}}=[\hat{c}_{1},\ldots,\hat{c}_{m}]^{\mathrm{T}}\)[13]. If \(\mathbf{Y}=\mathbf{BX}\), then by the covariance law, \(\mathbf{C_{Y}}=\mathbf{B}\mathbf{C_{X}}\mathbf{B}^{\mathrm{T}}\)[15] (Chapter 2). Thus, by the covariance law, the covariance matrix of \(\mathbf{\hat{c}}\) can be obtained as  
\[\mathbf{C_{\hat{c}}}=(\mathbf{\Phi}^{\mathrm{T}}\mathbf{P}\mathbf{\Phi})^{-1}. \tag{6}\]  
The design matrix \(\mathbf{\Phi}\) comprises sine and cosine basis functions of various cyclic frequencies \(\omega_{k}\in\Omega\) as well as other basis functions that characterize systematic noise (e.g., trends and offsets). The simplest case (out-of-context) is when \(\mathbf{c}=[c_{1},c_{2}]^{\mathrm{T}}\), and \(\mathbf{\Phi}\) is a \(n\times 2\) matrix whose columns are the cosine and sine basis functions of a cyclic frequency \(\omega_{k}\). In this case, the least-squares spectrum (LSS) of \(\mathbf{f}\) for \(\omega_{k}\) is defined as  
\[s(\omega_{k})=\frac{\mathbf{f}^{\mathrm{T}}\mathbf{P}\mathbf{\Phi}\mathbf{ \hat{c}}}{\mathbf{f}^{\mathrm{T}}\mathbf{Pf}}, \tag{7}\]  
which shows how much the wave of cyclic frequency \(\omega_{k}\) contributes to \(\mathbf{f}\). Note that \(s(\omega_{k})\) is in \((0,1)\). Aliasing may not occur in time series that are inherently unequally spaced [3; 16] underlining yet another important advantage of LSSA when applied to unequally spaced time series. LSSA can also consider constituents of known forms, such as trends, datum shifts, and sinusoidal basis functions of particular frequencies [14]. In order to find the underlying probability distribution function of LSS, one may assume that \(\mathbf{f}\) is a random vector of dimension \(n\) following the multidimensional normal distribution \(\mathcal{N}(\mathbf{0},\mathbf{C_{f}})\), where \(\mathbf{C_{f}}\) may be singular. Note that \(\mathbf{f}\) can also have any arbitrary mean that may be modeled by constituents of known forms [13].