# Time Series Anomaly Detection Using Signal Processing and Deep Learning
## 3 Methodology
### Model Training
#### 3.2.3 Fnn  
A Functional Neural Network [18; 54] is composed of a series of interconnected functional (continuous) neurons that are designed to process functional (time series) data. The input layer of the network takes in functional data, and each continuous neuron in the continuous hidden layer performs a non-linear transformation on the input data or values coming in from the previous layer. The output layer of the network produces a functional output that can be used for prediction, forecasting, or classification.  
Due to the success of the FNN and Bi-Functional Autoencoder (BFAE) [19] in time series data, we extend its application to anomaly detection. Here, the FNN Autoencoder is the BFAE, where the network architecture is similar to that of the traditional Autoencoderbut each component is a function rather than a scaler value. This is possible because of functional neurons which are given as follows:  
\[H_{(l)}^{(r)}(s)=\sigma\Big{(}b_{(l)}^{(r)}(s)+\sum_{j=1}^{J}\int w_{(l)}^{(r,j) }(s,t)H_{(l-1)}^{(r,j)}(t)dt\Big{)}\]  
where \(l\) indicates the layer and \(r,j\) are the neurons, \(l=1,2,3,\ldots,L\), \(\sigma(\cdot)\) is a non-linear activation function, \(b_{(l)}^{(r)}\) is the intercept function, and \(w_{(l)}^{(r,j)}\) is the bi-variate parameter function for the \(r\)th continuous neuron in the \(l\)th hidden layer coming from the \(j\)th continuous neuron of the \((l-1)\)th hidden layer.  
Now, let us see how this network functions using the continuous neurons defined above:  
* Encoder: The input layer that consists of time series data is transformed into a latent representation at an intermediate continuous hidden layer, denoted as the \(l^{\prime}\)th layer. This constitutes the encoder part of the FNN Autoencoder architecture, referred to by the functional encoder. The reduced-dimensional representation of the input in the \(l^{\prime}\)th continuous hidden layer is given as follows: \[\mathbf{Z}^{(r^{\prime})}(s)=H_{(l^{\prime})}^{(r^{\prime})}(s^{\prime}),\] where \(r^{\prime}=1,2,\ldots,R^{\prime}\) and \(R^{\prime}<R\). Here, the function \(\mathbf{Z}^{(r^{\prime})}\) is observed at \(S^{\prime}\leq S\) time points.
* Decoder: From this latent representation, the network works toward getting to the output layer, reconstructing the functional input values through continuous neurons in subsequent hidden layers. This constitutes the decoder part of the network, referred to as the functional decoder. The reconstructed values of the input in the output layer are given as follows: \[\widehat{X^{(r)}(s)}=H_{(L)}^{(r)}(s).\]  
In summary, the functional encoder maps the input from an \(R\)-dimensional functional space to an \(R^{\prime}\)-dimensional functional space, and the functional decoder reconstructs the input by mapping back from the \(R^{\prime}\)-dimensional functional space to the original \(R\)-dimensional functional space. The latent representation \(\mathbf{Z}^{(r^{\prime})}(s^{\prime})\) offers considerable flexibility, allowing the number of functional features \(R^{\prime}\) and the number of observed time points \(S^{\prime}\) to be adjusted according to the specific task requirements.  
Among these models, MLP is the least computationally expensive due to its simple feedforward structure. LSTMs are more resource intensive, as they maintain hidden states across time steps, increasing training time. FNNs incur the highest computational cost due to their functional representation, but their expressiveness often allows for better performance even with shallower networks.