# Inter-Subject MEG Decoding for Visual Information with Hybrid Gated Recurrent Network
## 2 Proposed Method  
For many conventional methods, a multi-channel MEG sample with spatial-temporal structure must be reshaped as a feature vector that its original spatial/temporal structure will be corrupted. This maybe one of the main reasons for their low performance. To deal with the spatial-temporal structure of MEG signal, we proposed a hybrid gated recurrent network (HGRN) which is a modified recurrent network based on gate recurrent units (GRU). To our best knowledge, the HGRN is the first gated recurrent network for MEG decoding. Compared with the conventional methods, the HGRN can directly process multichannel MEG signal, extract its spatial-temporal features and decode it within one network. The gated recurrent unit (GRU) is a special network structure which utilizes multiple gating units to store or regulate information [27; 28; 29]. In a GRU structure, its gated units are only activated by the current inputs and the previous inputs. Compared with the other kind of recurrent network structure, the GRU consists of less parameters and is converged faster as well. The GRU can adaptively capture the dependencies of the input data on different time scales. Many researches showed that the deep networks based on GRU achieved high performance in the sequence processing tasks [30; 31; 32]. In Figure 2, the internal structure of GRU is shown. The output of GRU at time \(t\) is \(\mathbf{h}_{t}=\{h_{t}^{1},...,h_{t}^{j},...\}\) where \(j\) denotes the \(j\)-th element. The \(h_{t}^{j}\) can be obtained by linear interpolation between the previous output \(h_{t-1}^{j}\) and the current candidate output \(\tilde{h}_{t}^{j}\) as follow:  
\[h_{t}^{j}=(1-z_{t}^{j})h_{t-1}^{j}+z_{t}^{j}\tilde{h}_{t}^{j} \tag{1}\]  
where \(z_{t}^{j}\) is the \(j\) element of update gate \(\mathbf{z}_{t}\). The \(z_{t}^{j}\) determines the updated scale of the output of GRU. Here, the update gate \(z_{t}^{j}\) can be obtained by the following statement:  
\[z_{t}^{j}=\sigma(W_{z}\mathbf{x}_{t}+U_{z}\mathbf{h}_{t-1})^{j} \tag{2}\]  
where \(\mathbf{x}_{t}\) denotes the input value at time \(t\), \(\mathbf{h}_{t-1}=\{h_{t-1}^{1},...,h_{t-1}^{j},...\}\) is the previous output at time \(t-1\). As the traditional recurrent unit, the candidate output \(\tilde{h}_{t}^{j}\) is computed by:  
\[\tilde{h}_{t}^{j}=\tanh{(W\mathbf{x}_{t}+U(\mathbf{r}_{t}\odot\mathbf{h}_{t- 1}))^{j}} \tag{3}\]  
where \(\mathbf{r}_{t}=\{r_{t}^{1},...,r_{t}^{j},...\}\) is the reset gate, \(\odot\) is the Hadamard product (element-wise product), the \(\tanh(\cdot)\) is a nonlinear function which is computed by  
\[\tanh(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{4}\]  
Following the update gate, the reset gate \(\mathbf{r}_{t}\) is defined as  
\[r_{t}^{j}=\sigma(W_{r}\mathbf{x}_{t}+U_{r}\mathbf{h}_{t-1})^{j} \tag{5}\]  
When the reset gate is closed, i.e., \(r_{t}^{j}=0\), the GRU will forget the previous states, and read the current input sequence instead. Based on the update gate \(z_{t}^{j}\) and the reset gate \(r_{t}^{j}\), the GRU can remember and update its states which is helpful for adaptively capturing the dependencies of input sequences in different time scales. Based on the GRU structure, the proposed HGRN is built to adaptively capture the dependencies of MEG signals in different time scales. In Figure 3, the structure of the proposed HGRN is shown. The proposed HGRN consists of two GRU-based layers, two down-sampling layers and an output layer. In the GRU-based layers, there are two GRUs, the GRU1 and GRU2 layer. Located in two different layers, GRU1 and GRU2 consist of 100 and 10 output units, respectively. The GRU layers are able to extract spatial features at the current time point and the previous ones. Compared with the other kinds of network structures, the HGRN model with GRU layer can efficiently process spatial-temporal data with a simple structure. In the down-sampling layers, max pooling (Max-Pool) operation is applied [33]. The Max-Pool(5) indicates that the max activation value at 5 contiguous time points is considered as the sampled value.