## 3 Proposed BMI System
### Machine Learning Models  
The extracted features are used as input to four machine learning based classifiers: Multilayer Perceptron (MLP), Auto-encoder (AE), 1D-Convolutional Neural Network (1D-CNN), Support Vector Machine (SVM). MLP classifierMLP is the most common feed-forward neural network, that uses standard gradient backpropagation method [52] to minimize the difference between the estimated and target values. It typically consists of one input layer, one output layer and one or more hidden layers [53]. Here, three MLP models are developed: MLP\({}_{1}\) composed of 1 hidden layer with 20 hidden neurons; MLP\({}_{2}\) composed of 1 hidden layer with 10 units; MLP\({}_{3}\) composed of 2 hidden layers with 20 and 10 neurons, respectively. All the developed networks are trained for about \(10^{3}\) epochs on a laptop with a  
Figure 6: Filter coefficients for collecting HOS features: (**a**) 0\({}^{\circ}\) in horizontal direction, (**b**) 90\({}^{\circ}\) in vertical direction, (**c**) \(-45^{\circ}\) and 45\({}^{\circ}\) in diagonal directions. 3.1 GHz Intel Core i5 processor with 8 GB memory installed. Note that the saturating linear transfer function is used as activation (since it provided good classification results) and that all MLP models end with a softmax output layer for performing the binary classification task (shadow vs. non-shadow). As an example, Figure 7 shows the MLP\({}_{1}\) architecture. _AE classifier_: AE model is trained through unsupervised learning algorithm and aims at generating the original data from the compressed representation of the input by an encoder-decoder operation [54]. Notably, the encoder compresses the input pattern (\(x\)) into a lower dimensional space (\(h\)):  
\[h=f(xW+b) \tag{12}\]  
where \(W\) is the weight matrix, \(b\) the bias vector and \(f\) the activation transfer function for the encoder. The decoder attempts to reproduce the original data from the compressed representation:  
\[\bar{x}=\bar{f}(h\bar{W}^{T}+\bar{b}) \tag{13}\]  
where \(\bar{f}\), \(\bar{W}\), \(\bar{b}\) are the corresponding activation function, weight matrix, bias in the decoder module. In this study, for fair comparison, three AE, denoted as AE\({}_{1}\), AE\({}_{2}\), AE\({}_{3}\) with similar topology of MLP\({}_{1}\), MLP\({}_{2}\), MLP\({}_{3}\), respectively are developed. For example, AE\({}_{1}\) (whose architecture is 36:20:36, Figure 8a) extracts 20 features (from the 36-dimensional input vector) used as input to softmax layer (trained in supervised modality) to perform the 2-way classification. Next, fine-tuning technique is applied by training again the whole network depicted in Figure 8b and sized 36:20:2. Similarly, AE\({}_{2}\) and AE\({}_{3}\) were developed. Note that all AE classifiers used saturating linear transfer activation function and were trained for about \(10^{3}\) epochs. _1D-CNN classifier_: CNN is a deep learning architecture typically used in 2D-image classification or pattern recognition [55; 56; 57; 58; 59; 60]. However, 1D-CNN is also employed to process 1D-patterns for discrimination purposes [61]. A common CNN includes different processing layers of _convolution_, _activation_, _pooling_, followed by a feed-forward MLP with softmax output layer. Notably, the first layer is composed of a set of \(S\) filters that computes the dot product with a local input region selected by the filter. Every filter moves with a step size \(s\), performing the convolution operation. \(S\) features maps are produced. The second layer is typically a rectified linear unit (ReLU) f(x) = max (x,0), employed for its effectiveness, simplicity and also because it provides non upper-bounded output values. The third layer performs the pooling operation. In particular, a filter moves along the input features maps (extracted by the previous layer) and estimates the maximum or average value. The output is a downsampled representation od the input data. It is to be noted that, here, the max pooling operation is used for its good translational-invariant properties [62].