# Anomaly Detection in Financial Time Series by Principal Component Analysis and Neural Networks +
## 2 The PCA NN Anomaly Detection Approach
### Contaminated Time Series Identification
#### 2.4.2 Neural Network Approach  
In view of obtaining a clearer separation of the densities, we propose an alternative approach for the computation of the anomaly scores built upon the reconstruction errors. Let \(F\) denote the function that associates an anomaly score with each reconstruction error \(\varepsilon^{i}\). In the naive approach, \(F\) corresponds to the \(\ell^{2}\)-norm. Hereafter, \(F\) instead represents the outputs of a feedforward neural network (NN). The training of the corresponding NN aims to minimize a loss function that reflects our ambition to construct a function \(F\), giving accurate anomaly scores. The network used for computing the anomaly scores is defined by  
\[F(\varepsilon)=\left(h^{H}\circ h^{H-1}\circ\ldots h^{2}\circ h^{1}\right)( \varepsilon),\]  
where \(h^{i}(\varepsilon)=\left(W^{i}\cdot\varepsilon+b^{i}\right)^{+}\) represents the computation carried out in the \(i\)-th layer of the neural network with \(n_{h^{i}}\) hidden units. \(H\) stands for the number of layers of the NN, with  
Figure 1: Empirical densities of anomaly scores given by the naive approach for uncontaminated time series in black and contaminated time series in red in a training set (**left**) and test set (**right**). The dotted dark red lines represent the cutoff values. ReLU activation applied element-wise. We finally estimate the labels from the outputs following \(\hat{\mathbf{A}}=\mathbb{1}_{\{(F(\epsilon)-s)^{+}>0\}}\). The idea of the learning is to calibrate the weights \(\mathbf{W}:=\left\{W^{i}\in\mathbb{R}^{n_{h}^{i}\times n_{h}^{i-1}},i=2,\ldots,H\right\}\) and the biases \(\mathbf{b}:=\left\{b^{i}\in\mathbb{R}^{n_{h}^{i}},i=1,\ldots,H\right\}\) so that the NN is able to accurately assess the extent to which a time series is contaminated, with a clear distinction between the anomaly scores assigned to uncontaminated time series and those assigned to contaminated ones while integrating the calibration of the cutoff value \(s\) as part of the learning. We use \(\Theta:=\{\mathbf{W},\mathbf{b},s\}\) to denote the set of parameters to be calibrated through the NN training. To meet these needs, the loss we minimize during the learning is given by  
\[\mathcal{L}_{\Theta}\big{(}\mathbf{A},\hat{\mathbf{A}}\big{)}=\text{BCE} (\mathbf{A},\hat{\mathbf{A}})+\text{AUCDensity}^{u}_{\mathbf{A},F(\epsilon)}+\text{AUCDensity}^{c}_{\mathbf{A},F( \epsilon)^{\prime}} \tag{5}\]  
where  
\[BCE(\mathbf{A},\hat{\mathbf{A}})=-\frac{1}{n}\sum_{i}\Bigl{(}A^{i}\log( \hat{A}^{i})+(1-A^{i})\log(1-\hat{A}^{i})\Bigr{)}\]  
is the binary cross-entropy, a well-known loss function classically used for classification problems. To this first component of our loss function, we add two components that aim to downsize the overlapping region between the densities of the anomaly scores of both types of observations. In order to have control of the latter, we consider AUCDensity\({}^{u}\) and AUCDensity\({}^{c}\), which correspond to the area under the curve of the probability density function of the anomaly scores the model assigns to contaminated and uncontaminated observations, respectively, i.e.,  
\[\text{AUCDensity}^{u}(s)=\int_{s}^{\infty}\hat{f}^{u}_{\mathbf{A},F( \epsilon)}(\omega)d\omega\,,\qquad\qquad\text{AUCDensity}^{c}(s)=\int_{- \infty}^{s}\hat{f}^{c}_{\mathbf{A},F(\epsilon)}(\omega)d\omega\,. \tag{6}\]  
The bounds of these integrals depend on the cutoff value \(s\) and define the region for which we want the probability density functions to be as small as possible, which allows us to estimate \(\hat{s}\). The estimated probability density functions \(\hat{f}^{u}_{\mathbf{A},F(\epsilon)}\) and \(\hat{f}^{c}_{\mathbf{A},F(\epsilon)}\) depend on \(F(\epsilon)\), the scores assigned by the contaminated time series identification model, and the identification labels \(\mathbf{A}\). In Algorithm 2, we describe the scoring and cutoff value calibration achieved by the calibration of a feedforward network with the loss (5).