# Deep Representation of EEG Signals Using Spatio-Spectral Feature Images
## 4 Results
### Deep Representation
In an SSFI pattern, dead channels are useful for producing features that depend on a subset of frequency bands, while the others are excluded. For example, the first pattern in Figure 8b shows that the corresponding filter in layer CNV2 produces an output that uses information from the \(\delta\), \(\beta\), and \(\gamma_{1}\) frequency bands only. This suggests that layers CNV2 and CNV3 produce features that focus on different frequency bands. Interestingly, the number of dead channels decreases for layer CNV4 and there are almost none in layer CNV5, as shown in Figure 9. This indicates that CNV4 and CNV5 layers do combine features from all the channels in the previous layers, i.e., no channel is being filtered out. From Figure 9, it can be concluded that layers CNV4 and CNV5, which produce high-level features, focus on more localized areas of the brain. This contrasts with the low-level features produced by CNV1, which focus on larger regions. Figure 9: Selection of typical SSFI deep representation patterns corresponding to four filters at layers (**a**) CNV4 and (**b**) CNV5 for subject S5. Patterns are shown as rows, frequency bands as columns. To identify the association of brain regions and auditory task in the PhyAAt dataset, we further computed SSFI saliency maps using the Vanilla and SmoothGrad approaches. We extracted the top five SSFI arrays of subject S5 that were correctly classified by the CNN model trained on S5 and produced the saliency maps for each task (listening, writing, and resting). Saliency maps are shown in Figure 10. Unlike the conventional approach of displaying saliency maps over input images, Figure 10 displays SSFI saliency maps alone to highlight the relevant regions of the brain. This is a suitable visualization in our scenario, since the topography remains constant. Figure 10 reveals that the CNN model under investigation is looking into roughly the same region of the brain in each task, i.e., listening, writing, and resting, with slight variations. SmoothGrad saliency maps suggest that during a listening task, the CNN model focuses more on the right side of the brain. To identify the common brain regions where each CNN model focuses to produce a correct prediction, we averaged SSFI saliency maps corresponding the top the 100 SSFI input arrays. A similar approach was used in [43] to localize the importance of an ECG signal in a DNN study. The averaged saliency maps during listening tasks for all the 25 subjects are shown in Figure 11, and allow us to identify the region of the brain that each CNN model is attending to during a listening task. Average saliency maps indicate that the brain regions associated with listening tasks are different in each subject, which might explain why models trained on one subject might perform poorly on other subjects [4]. Identifying subjects such that CNN models trained on them use common brain regions could be useful. Interestingly S1, S2, and S25 exhibit the same region of brain activity, and so do S3 and S4. A detailed analysis of these patterns could potentially reveal the reason why a CNN model trained on one subject performs differently on different subjects. This approach could also allow us to explore the association of brain regions with different brain activities. Figure 10: Saliency maps for top five SSFI input arrays for listening (L), writing (W), and resting (R) tasks that were classified correctly. Saliency maps are generated using (**a**) Vanilla (**b**) SmoothGrad approach.