# Anomaly Detection in Financial Time Series by Principal Component Analysis and Neural Networks +
## 2 The PCA NN Anomaly Detection Approach
### Theoretical Basics of Principal Component Analysis  
A fundamental intuition behind our anomaly detection algorithm is the existence of a lower dimensional subspace, where normal and abnormal observations are easily distinguishable. Since the selected principal components explain a given level of the variance of the data, they represent the common and essential characteristics of all observations. As one might reasonably expect, these characteristics mostly represent the normal observations. When the inverse transformation is applied, the model successfully reconstructs the normal observations with the help of the extracted characteristics, although it fails at reconstructing the anomalies.  
In this section, we recall the theoretical tenets of principal component analysis (PCA) that are useful for our purpose. The idea of PCA is to project the data from the observation space of dimension \(p\) into a latent space of dimension \(k\), with \(k<p\). This latent space is generated by the \(k\) directions for which the variance retained under projection is maximal.  
The first principal component \(U\) is given by \(U=w^{\top}\mathbf{X}\), where \(w\) is a to-be-determined vector of weights. Since PCA aims to maximize the variance along its principal components, \(w\) is searched for as  
\[\arg\max_{\left\lvert w\right\rvert=1}w^{\top}\Sigma w\,,\]  
where \(\Sigma\) is the covariance matrix of \(\mathbf{X}\). The optimal solution by the Lagrangian method is given by \(\Sigma w=\lambda w\), where \(\lambda>0\) is a Lagrangian multiplier. Multiplying both sides by \(w^{\top}\) results in \(w^{\top}\Sigma w=\lambda\). Following the same process with an additional constraint regarding the orthogonality between the principal components, one can show that the \(k\) first principal components correspond to the eigenvectors associated with the \(k\) dominant eigenvalues of \(\Sigma\).  
The transfer matrix \(\Omega_{\mathbf{X}}\in\mathbb{R}^{k\times p}\) is defined by the eigenvectors of \(\Sigma\) associated with its \(k\) largest eigenvalues. This transfer matrix \(\Omega_{\mathbf{X}}\) applied to any observation in the original observation space projects it into the latent space. Since the transformation is linear, reconstructing the initial observations from their equivalent in the latent space is straightforward. Using PCA on the data matrix \(\mathbf{X}\), the transfer matrix \(\Omega_{\mathbf{X}}\) can be inferred. The projection of the observations into the latent space \(\mathbf{Z}\in\mathbb{R}^{n\times k}\) is then given by \(\mathbf{Z}=\mathbf{X}\Omega_{\mathbf{X}}^{\top}\), whereas their reconstructed values are given by \(\widehat{\mathbf{X}}=\mathbf{Z}\Omega_{\mathbf{X}}\).  
The reconstruction errors \(\varepsilon^{i}\) for each time series defined by  
\[\varepsilon^{i}=\widehat{X}^{i}-\bar{X}^{i},\quad i=1,\ldots,n, \tag{3}\]  
referred to in the sequel as the new features, are our new representation of data, given as inputs to our models. Algorithm 1 summarizes how to compute these reconstruction errors. The two steps of the model use the same inputs, namely these reconstruction errors. Nevertheless, the processing these inputs undergo in each step differs.  
```
inputs:number of principal components \(k\)  out of sample data \(\mathbf{X}^{Test}\)  training data if not trained yet \(\mathbf{X}^{Train}\) ifnot trained yetthen \(\Omega_{\mathbf{X}^{Train}}\leftarrow\text{PCA}(\mathbf{X}^{Train},k)\)  end if \(\mathbf{\varepsilon}^{Train}=\mathbf{X}^{Train}\Big{(}\Omega_{\mathbf{X}^{Train}}^{\top} \Omega_{\mathbf{X}^{Train}}-I_{p}\Big{)}\) \(\mathbf{\varepsilon}^{Test}=\mathbf{X}^{Test}\Big{(}\Omega_{\mathbf{X}^{Train}}^{\top} \Omega_{\mathbf{X}^{Train}}-I_{p}\Big{)}\) outputs:\(\varepsilon^{Train}\) reconstruction errors for the train set \(\mathbf{\varepsilon}^{Test}\) reconstruction errors for the test set
```  
**Algorithm 1** Feature extraction with PCA