## Materials and Methods
### Causal Feature Extractor  
In the encoder-decoder architecture, both the encoder and the decoder are independent modules that can be implemented in different ways. For example, they can consist of Convolutional Neural Networks (CNN) [19], which is a typical selection method in images. In the case of sequential data, Long Short-Term Memories (LSTM) [20] are more frequently found. As mentioned before, the purpose of this study is to analyze the feasibility of a new type of layer for the encoder, called Causal Feature Extractor (CFE) [8]. This method is inspired by the Dilated Convolutional Neural Networks and the Causal Convolutional Neural Networks, introduced by [PERSON] et al. [9]. The proposed model, depicted in Figure 1b, is built under three main ideas:  
* First, in order to extend the receptive field of the convolutions without requiring large kernels, several convolutional neural layers are stacked, and each one has two times the dilation of the previous one. That is, in the first layer, the convolution for position \(t\) depends on \(t,t-1,t-2\ldots\); in the second layer, it depends on \(t,t-2,t-4\ldots\); in the third layer, \(t,t-4,t-8\ldots\), and so on. * Second, with the aim of making a better use of the attention mechanisms in comparison with CNNs, these stacked convolutional layers are turned into causal convolutions, meaning that the output at one position will depend on the inputs previous or next to that position, but never both. This is the same idea as the Causal CNN proposed by [PERSON] et al. [9]. * Third, considering that the use of causal layers means the misuse of one part of the input, two stacks of causal convolution layers are used, each one taking into account a different direction of the input (the previous or the subsequent input values). The same idea of bidirectionality has also been applied to LSTMs [21]. The main hyperparameters that define the structure of the CFE encoder are the CNN kernel width, the desired receptive field, and the number of features to generate, \(f\). The first hyperparameter  
Figure 1: Scheme of the proposed neural network architecture. (**a**) Global scheme of the encoder-decoder architecture including the attention mechanism. The input matrix, \(\mathbf{X}\), which contains \(l\) vector elements \((x_{1},x_{2}\ldots x_{l})\), is transformed into an intermediate code, \(\mathbf{Z}=z_{1},z_{2}\ldots z_{l}\). Then, the attention mechanism selects the importance, \(a\), of each tuple at each time, \(t\). Using both values, the decoder produces the output at each instant, \(y_{t}\). The hidden state of the decoder at the previous instant, \(h_{t-1}\), is fed into the attention mechanism and into the decoder. \(<\)_sos\(>\)_ means “start of sequence”. (**b**) Outline of the proposed Causal Feature Extractor for the part of the encoder, in this case with 3 layers. The input is matrix \(\mathbf{X}\), with \(l\) vectors of size \(v\). There are two independent sub-nets (upper and lower), each of which generates \(f/2\) features for each input vector. They are convolutional NNs which are causal (\(a\) uses the previous values, and \(b\) uses the later values) and dilated (the step in the 1 st layer is 1, in the 2 nd layer 2, and in the 3 rd layer 4). The output is the code \(\mathbf{Z}\), with \(f\) features for each input vector. indicates the width of the kernels of the convolutions. Along with the second parameter, they determine the number of layers of the CNN. For example, if the kernel width is 5 and the desired receptive field is 20, then there would be 3 convolutional layers (since the dilations are multiplied by 2, the receptive fields of the 1 st, 2 nd and 3 rd layers would be 5, 10 and 20, respectively). Other hyperparameters that are used during the training process are the size of the batches applied in the input, the way of normalizing the weights of the convolutions, the maximum norm of the gradient, and the dropout rate applied to the neurons; there is also the possibility of including or not the position of the input values in the encoder.