# 4-Class MI-EEG Signal Generation and Recognition with CVAE-GAN
## 3 Methods
### CVAE-GAN Construction  
CVAE models a generator as a pair of encoder and decoder networks. Compared to VAE, CVAE is able to generate data based on certain attributes. The encoder learns a latent representation \(z\) from the data \(x\) on condition of the \(y\) belonging to a specific category, while the decoder aims to predict the specific category according to the learned representation z. The generated data will be distorted owing to the deficiency of input and output. Fortunately, the generated performance will be improved when it combines with the discriminator. On the other side, the generator of GAN is isolated from real EEG, thus the introduction of the encoder will make GAN more stable. The whole CVAE-GAN architecture is shown in Figure 2.  
Considering the goal of generating process under the control of the specified category, label dates were extra input to the encoder and decoder. We feed the training samples and its corresponding label to the encoder, then concatenate the hidden representation with the corresponding label and feed it to the decoder to train the network. Ultimately, we could generate data with the specified label by feeding the decoder with the noise sampled from the Gaussian distribution and the assigned label after the training process. Therefore, the CVAE can be formulated as:  
\[L_{G}=\sum_{i}\Bigl{\{}(x_{i}-\widetilde{x_{i}})^{2}+0.5[\sum(x_{i}|y_{i})\,+ \mu^{2}(x_{i}\Bigl{|}y_{i})-1-\log\sum(x_{i}|y_{i})\,]\Bigr{\}}. \tag{2}\] \begin{table}
\begin{tabular}{c c c} \hline \hline
**Datasets** & **Private** & **Public** \\ \cline{2-3}  & **D\({}_{1}\)** & **D\({}_{2}\)** \\ \hline Subject & 3 & 9 \\ Sample rate (Hz) & 250 & 250 \\ Imagery task & left, right hand, tongue, and both feet & 1 left, right hand, tongue, and both feet \\ MI periods for processing & 2s & 2s \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Properties of raw materials. We employed the CNN as the implementation architecture of the encoder and decoder. Their mapping details are illustrated in Table 2.  
The input images were convolved with the kernels to be trained and were then put through to generate the map of output in the convolution layer [26]. In a given layer, the map of \(k\)-th features can be obtained as:  
\[H_{ij}^{k}=f(a)=f(W^{k}*x)_{ij}+b_{k}, \tag{3}\]  
\[f(a)=\text{ReLU}(a)=\ln(1+\text{e}^{a}), \tag{4}\]  
where 20 and 40 kernels were employed in the first and second layer, respectively. The max-pooling layer is a completely connected layer for different dimensions. The feature mapping after convolutional kernel was down-sized into a smaller sample in the pooling layer. With the proposed approach, the network is fed with the labeled training set, and the error E is computed taking into account the fact that the desired output is different from the output of the network. Subsequently, the stochastic gradient descent algorithm [27] was applied to minimize the error occurring with changes in the network and thus to optimize the network weights and filters in the convolutional layer. The output of the encoder formed vector \(z\) through a two layers CNN, which extracts the temporal, spectral, and spatial features from the 3D power image representation of MI-EEG. The encoding vector h was mapped into two feature compression layers in order to calculate the mean \(\mu\) and variance \(\sigma\) of the neural networks. The latent representation \(z\) is calculated as:  
\[z=\mu+\sigma\odot\epsilon, \tag{5}\]  
where \(\epsilon\) means randomly sampling from standard normal distribution N(0,1) and \(\odot\) denotes an element-wise product. The covariance matrix of \(z\) is diagonal. Because we expected that the components of latent vector \(z\) should be mutually independent so as to makes them informative. After every convolution operation, we adopted batch norm to improve the model training efficiency and nonlinearity. The 50% dropout rate was used to prevent from overfitting.