## 4 Pattern Recognition
### Restricted Boltzmann Machines  
A special type of ANN is the one developed by [PERSON] [55], known as Restricted Boltzmann Machines (RBM). RBMs are two-layer neural networks of stochastic units, divided into _visible_ units \(\textbf{v}=(v_{1},\cdots,v_{i})\) and _hidden_ units \(\textbf{h}=(h_{1},\cdots,h_{j})\), which have symmetrical connected weights (see Figure 8). The visible units represent the data, while the hidden units are known as feature extractors. RBMs pretend to be model dependencies  
Figure 7: Backpropagation applied on a Neural Network. over visible variables. The probability \(p(\mathbf{v},\mathbf{h};\Theta)\propto e^{-E(\mathbf{v},\mathbf{h};\Theta)}\) is known as the Boltzmann distribution, which has an energy function described as:  
\[E(\mathbf{v},\mathbf{h};\Theta)=-\mathbf{h}^{T}\mathbf{W}\mathbf{v}-\mathbf{b}^ {T}\mathbf{v}-\mathbf{c}^{T}\mathbf{h} \tag{11}\]  
with \(\mathbf{W}\) as the symmetric weights, \(\mathbf{b}\) and \(\mathbf{c}\) as the bias of the visible and hidden units, respectively, and \(\Theta=(\mathbf{W},\mathbf{b},\mathbf{c})\). The two conditional distributions over the variables, i.e., hidden given the visible and visible given the hidden, are given by:  
\[p(h_{j}=1|\mathbf{v})=\sigma_{h}(c_{j}+\sum_{i}v_{i}w_{ij}) \tag{12}\]  
and  
\[p(v_{i}=1|\mathbf{h})=\sigma_{v}(b_{i}+\sum_{j}h_{j}w_{ij}) \tag{13}\]  
where \(\sigma\) represents the activation function. Since the hidden variables cannot be observed, we need an algorithm that improves the RBM representation of the system. This algorithm is called Contrastive Divergence (CD) [56], which allows fitting the probability \(p(v)\) to a certain set of observations (e.g., EEG signals). The pseudo code for RBM can be seen in Algorithm 5. ```
% Notation: \(x\gets b\) means \(x\) is set to value \(b\) % \(x\sim p\) means \(x\) is sampled from \(p\) Input network:  Training pair \(v=(x_{i},y_{i})\)  Learning rate \(\alpha\) Init network:  Train network: \(v^{(0)}\gets v\)  Loop Gibbs sampling \(t=0,\ldots k-1\) % Positive phase \(\hat{h}^{(t)}\leftarrow\sigma\Big{(}c+Wv^{(t)}\Big{)}\) % Negative phase \(h^{(t)}\sim p(h|v^{(t)})\) \(v^{(t+1)}\sim p(v|h^{(t)})\) end Loop \(\hat{h}^{(}k)\leftarrow\sigma\Big{(}c+Wv^{(k)}\Big{)}\) % Updates for \(\theta\in\Theta\)do \(\theta\leftarrow-\alpha\Big{(}\frac{\partial}{\partial\theta}E(v^{0},\hat{h}^ {0})-\frac{\partial}{\partial\theta}E(v^{k},\hat{h}^{k})\Big{)}\) end for Output network: Return weights and biases

Figure 8: A representation of the RBM with connections symmetrically connected weights. ### Advantages and Disadvantages of the Methods

It is intended to test the efficacy of the proposed methods for BCI, but, before doing that, it may be convenient to clarify the advantages and disadvantages of each method. Table 1 is a listing of the main characteristics of each of them. ### Accuracy and Cross Validation

In general, it is important to know how well a given method performs over a specific task; thus, it is important to calculate its accuracy. Therefore, to calculate the accuracy, the Mean Square Error (\(MSE\)) was used, as shown in Equation (14), with \(t_{i}\) and \(y_{i}\) as the observed and predicted outputs. \[Acc =100-MSE,\quad\text{with}\] \[MSE =\frac{100\%}{n}\sum_{t=1}^{n}(t_{i}-y_{i})^{2} \tag{14}\]

Another important measure to be done is how the classification method is going to behave when dealing with independent data, i.e., how general the method is. It is important to check that the method does not overfit, which means that it obtains a perfect score when dealing with training data but has a poor performance when it is exposed to unseen data. One way to overcome this problem is to observe the performance of the classifier over a training dataset and then verify it using a test dataset; this is the basic idea behind a technique called _cross validation_. However, we still have the problem that the behavior of the system may depend heavily on which data points are used for training and which ones are used for training.