# Deep Representation of EEG Signals Using Spatio-Spectral Feature Images
## 3 Methods
### Deep Representation Analysis  
Activation maximization is a common approach to investigating deep representations in DNNs and consists in identifying the input patterns that activate the neurons in each trained layer. The input patterns that activate the neurons in a given layer can be interpreted as those that the layer is focusing on. Therefore, a deep representation can be interpreted in terms of its corresponding maximization patterns. In problems involving images of the natural world, such as faces and objects, this approach produces patterns which can be meaningfully interpreted, such as eyes in a DNN model trained for face recognition, or wheels in a DNN model trained for object recognition. The application of activation maximization to problems involving EEG signals is more challenging, as they lack patterns that posses the interpretability available in images of physical objects, speech signals, or text.  
Our approach to analyzing deep representations in a DNN trained on EEG signals produces SSFI patterns. In other words, in our approach, deep representations in a DNN are interpreted by identifying the areas of the brain and the frequency bands that activate the associated layers. We assume that a processing pipeline transforms multi-channel EEG signals into SSFI, which are then fed to a trained DNN, similar to the one presented in Figure 1. Therefore, rather than looking for temporal patterns in EEG signals that activate the layers of a DNN, our approach looks for spatio-spectral patterns. Given a DNN trained on SSFIs, activation maximization patterns are generated by feeding random, noisy images to a trained network and maximizing the activation of a selected deep neuron by updating the input image using the gradient ascent method [40]. Since our input image is not a conventional three-channel RGB image, we display a pattern for each input channel separately. In this study, we train 25 separate CNN models that share the same architecture on SSFI arrays from multi-channel EEG signals from 25 subjects. Comparing the SSFI patterns activating each layer allows us to explore which brain areas and frequency bands are involved in each auditory task for each subject, which can suggest opportunities for transfer learning and shed light into common and individual brain mechanisms.  
A second approach to analyzing a trained DNN is to generate saliency maps [41]. In this approach, one of the input images is fed to the DNN, and by computing a gradient, a map is produced that indicates the spatial locations of an input image that are important for computing the output probability score. Saliency maps are usually overlaid on their corresponding input image to identify the areas of the image that are useful in producing the final prediction. Assuming that the input to a DNN consists of SSFIs generated from multi-channel EEG signals, in this study, we create SSFI saliency maps, which identify the brain areas that are relevant for each prediction. Specifically, we focus on those SSFI images from the training set that have the highest probability score for its true class and we generate averaged saliency maps to identify common brain regions that are associated with the listening task.