 The core idea is to learn plausible representations through the combined use of human neural activity evoked by natural imagesas a supervision mechanism for deep learning models. To accomplish this, we propose a multimodal approach that uses two different deep encoders, one for images and one for EEGs, trained in a siamese configuration  for learning a joint manifold that maximizes a compatibility measure between visual features and brain representation.
<meaning-changed>  The core idea is to learn plausible representations through the combined use of human neural activity evoked by natural imagesas a supervision mechanism for deep learning models. To accomplish this, we propose a multimodal approach that uses two different deep encoders, one for images and one for EEGs, trained in a siamese configuration  for learning a joint manifold that maximizes a compatibility measure between visual features and brain representation.
 The core idea is to learn plausible representations through the combined use of human neural activity and natural images. Thus, we first propose a model, EEG-ChannelNet, to learn a brain manifold for EEG classification. After verifying that visual information can be extracted from EEG data, we introduce a multimodal approach that uses two different deep encoders, one for images and one for EEGs, trained in a siamese configuration for learning a joint manifold that maximizes a compatibility measure between visual features and brain representation.