# Identification of Epileptic EEG Signals Using Convolutional Neural Networks
## 2 Convolutional Neural Networks (CNNs)  
A CNN is a multilayer neural network (NN) architecture which incorporates one or more convolution, max-pooling and fully-connected layers. The convolution layers represent the core building block of the network and have a hierarchical structure. In this paper, a one-dimensional CNN is used for the detection of epilepsy. Figure 1 depicts the structure of the one-dimensional double CNN. The input of the first convolution layer represents the input space and the output represents the feature map. The input and output of the subsequent convolutional layers are feature maps of input space. In Figure 1, we explore the four convolutional layers. The complex features of input space are represented by using the stacked hierarchical structure of convolutional layers. Features obtained from the convolutional layers are fed to the pooling layers. A rectified linear unit (ReLU) activation function is applied to the obtained feature map. In this layer, the relevant features are retained and the rest are discarded. The obtained features are transformed into a one-dimensional array known as a feature vector. The feature vector is a one-dimensional array and is the input for the fully connected network. This layer calculates the output of the CNN [48; 49]. Considering the formulation of the above operations, and using the input signal and local kernels, the output of the convolution layer will be computed. Each output is calculated by the dot product of input and kernel (filter) weight coefficients. Depending on the number of kernels, the volume will be [\(p\),\(q\)]. Here, \(q\) is a number of kernels. The feature maps are determined by using kernels (filters). For \(x_{i,j}^{l}\) input space, the feature value \(z_{i,j,k}^{l+1}\) at the location (\(i\), \(j\)) in the \(k\)th feature map of the \(l+1\)th convolutional layer is calculated by:  
\[z_{i,j,k}^{l+1}=\mathbf{w}_{k}^{l^{T}}\cdot\mathbf{z}_{i,j}^{l}+b_{k}^{l} \tag{1}\]  
where weight \(\mathbf{w}_{k}^{l}\) is the vector and \(b_{k}^{l}\) is the bias term of the \(k\)th filter of the \(l\)th layer. In the first layer, \(l=1\) and \(\mathbf{z}_{i,j}^{l}=\mathbf{x}_{i,j}^{l}\). The kernel generates the feature map \(z_{i,j,k}^{l+1}\) as shown in [50; 51; 52]. The second convolutional operation is applied to the output of the first layer. Unlike multi-layer networks, the activation function is used for non-linear transformation of signals of CNN to detect non-linear features. For this aim, the non-linear activation function is applied for the transformation of (1). \[\mathbf{a}_{i,j,k}^{l+1}=a\Big{(}\mathbf{z}_{i,j,k}^{l+1}\Big{)} \tag{2}\]  
where \(\mathbf{z}_{i,j,k}^{l+1}\) is calculated by the formula (1). Typical activation functions are tanh, sigmoid and ReLU [52]. In the paper, the non-saturated activation function called ReLU is utilized. It is denoted by \(max\Big{(}0,\mathbf{z}_{i,j,k}^{l+1}\Big{)}\) ReLU prunes the negative values to zero and holds positive values. ReLU activation function produces quicker results than the sigmoid or tanh activation functions and speeds up execution time [50; 51; 52]. This function can induce the sparsity in the hidden units and obtain sparse representations of the network. The use of ReLU allows easy training of the deep networks. By reducing the feature map resolution, the pooling layer achieves shift-invariance. A pooling layer is often used between the convolutional layers. The feature map of a convolutional layer is  
Figure 1: One-dimensional convolutional neural network (CNN). converted to the feature map of a pooling layer. For each feature map \(a_{i,j,k}^{l}\) the pooling layer output is determined by:  
\[y_{i,j,k}^{l+1}=pool\big{(}a_{m,n,k}^{l+1}\big{)},\forall(m,n)\in R_{i,j} \tag{3}\]  
where \(R_{i,j}\) is a local neighbourhood around the location \((i,j)\). In the literature, average pooling and max pooling are often used as typical pooling operations. In this paper, max pooling is used. As a result of using the above operations, the feature map will be obtained.