## 2 Materials and Methods
### Stacked Sparse Autoencoders  
AEs are deep networks trained in an unsupervised fashion to replicate the input at the output [61]. AEs consist of an encoder and a decoder. An encoder maps an input x to a new representation z, which is decoded back at the output to reconstruct the input \(x^{\prime}\):  
\[Z=h\left(Wx+b\right) \tag{1}\]  
\[X^{\prime}=g\left(W^{\prime}z+b^{\prime}\right) \tag{2}\]  
where \(h\) and g are activation functions, \(W\) and \(W^{\prime}\) are weight matrices, and \(b\), \(b^{\prime}\) are bias vectors for the encoder and decoder, respectively [62]. The error between the input k and the reconstructed input \(x^{\prime}\) is optimized as follows:  
\[\min_{\left(W,b,W^{\prime},b^{\prime}\right)}=\sum_{i=1}^{n}||x_{i}-x^{\prime }_{i}||^{2} \tag{3}\]  
Figure 2: Rectified EMG recorded from the six surface electrode systems in a trial of an intact-limb subject. Six iEMG channels were also recorded concurrently (not shown). Eleven movements (including rest) were repeated four times with a contraction and relaxation time of five seconds. A group of four with the same gray level represents the four repetitions of the same movement. In this work, stacked sparse autoencoders (SSAE) [57] were used with two hidden layers, consisting of 24 and 12 hidden units (Figure 3). For both layers, the logistic sigmoid and linear functions were used for the encoders and decoders, respectively. In SSAE, the output of one AE is fed to the input of another AE [39] and sparsity is encouraged by adding regularization to the cost function [63], which stands for the average output activation of a neuron. An average output activation for a neuron \(i\) can be formulated as:  
\[\hat{p_{i}}=\frac{1}{n}\cdot\sum_{j=1}^{n}z_{i}\big{(}x_{j}\big{)} \tag{4}\]  
where \(i\) is the \(i\)th neuron, \(n\) is the total number of training examples, and \(j\) is the \(j\)th training example. This regulariser is introduced to the cost function using the Kullback-Leibler divergence [64]:  
\[\Omega_{sparsity}=\sum_{i=1}^{d}p\log\left(\frac{p}{\hat{p_{i}}}\right)+(1-p) \log\left(\frac{1-p}{1-\hat{p_{i}}}\right) \tag{5}\]  
where \(d\) is the total number of neurons in a layer [65] and \(p\) is the desired activation value, called sparsity proportion (SP). An L2 regularization term (L2R) is further added to the cost function to control the weights:  
\[\Omega_{weights}=\frac{1}{2}\sum_{l}^{L}\sum_{j}^{N}\sum_{i}^{K}\left(w_{ji}^ {(l)}\right)^{2} \tag{6}\]  
where \(L\) represents the number of hidden layers, \(N\) is the total number of observations, and \(K\) is the number of features within an observation. Therefore, by inserting the regularization terms from Equations (5) and (6) into the reconstruction error in Equation (3), the cost function can be formulated as follows:  
\[E=\underbrace{\frac{1}{N}\sum_{n=1}^{N}\sum_{k=1}^{K}(x_{kn}-\hat{x}_{kn})^{2 }}_{mean\text{ \emph{square error}}}+\lambda* \underbrace{\Omega_{weights}}_{L2}+\beta* \underbrace{\Omega_{sparsity}}_{Sparsity} \tag{7}\]  
\[Regularization\hskip 56.905512 ptRegularization\hskip 56.905512 pt(SR)\]  
Figure 3: Block diagram of stacked sparse autoencoders (SSAE) used in this work. Features at layer 1 were improved by minimizing the error using Equation (3). These improved features were then fed as input to the next AE and again, improved features at layer 2 were fed to the softmax classifier where labels were obtained. All layers were trained independently from each other and were stacked together. Hence, features were learned in an unsupervised fashion, whereas classification was supervised. The three optimization parameters are \(\lambda\) (coefficient for L2R), which prevents overfitting; \(\beta\) (coefficient for sparsity regularization SR) that controls the sparsity penalty term; and \(p\) (SP), which sets the desired level of sparsity [33; 66].