PDAML: A Pseudo Domain Adaptation Paradigm for Subject-independent EEG-based Emotion Recognition

Abstract: Domain adaptation (DA) and domain generalization (DG) methods have been successfully adopted to alleviate the domain shift problem caused by the subject variability of EEG signals in subject-independent affective brain-computer interfaces (aBCIs). Usually, the DA methods give relatively promising results than the DG methods but require additional computation resources each time a new subject comes. In this paper, we first propose a new paradigm called Pseudo Domain Adaptation (PDA), which is more suitable for subject-independent aBCIs. Then we propose the pseudo domain adaptation via meta-learning (PDAML) based on PDA. The PDAML consists of a feature extractor, a classifier, and a sum-decomposable structure called domain shift governor. We prove that a network with a sum-decomposable structure can compute the divergence between different domains effectively in theory. By taking advantage of the adversarial learning and meta-learning, the governor helps PDAML quickly generalize to a new domain using the target data through a few self-adaptation steps in the test phase. Experimental results on the public aBICs dataset demonstrate that our proposed method not only avoids the additional computation resources of the DA methods but also reaches a similar generalization performance of the state-of-the-art DA methods.

Reviews and author responses:

The authors present an incremental improvement to previously published approaches by Zhao et al., and their SEED dataset, which is currently not available, since a link provided in the manuscript lands on an error page (see below), so a provided code is also unfortunately useless: http://bcmi.sjtu.edu.cn/∼seed/index.html -> 404，您请求的文件不存在!
The authors use also the same hand-engineered features (why not end-to-end data-driven?) by Zhao et al., which is another copy and paste step from previous publications. The authors also seem to mix or not understand the difference between affect and induced emotion. The experimental SEED database used in the project, but not available anymore online, seems to be related to affect recognition from brainwaves (here EEG) in response to emotional videos, but not the subject's emotion itself, which seems to be not reported in the original experiment. Finally, a relation to BCI is not explained well. BCI is about intentional, and hopefully fast, control of computers or machines with modulated brainwaves. This manuscripts concerns only affect recognition in response to emotional videos.  The manuscript concerns an incremental improvement of previously published results by replacing a machine learning model with the proposed PDAML. The evaluated concept of domain adaptation is exciting but not very novel. The authors recycle previously published hand-engineered features to a questionable emotional response, rather affective EEG modulation. Why was no end-to-end data-driven approach tested? The proposed simple LSTM-based feature processor followed by a regular MLP classifier with a DA-focusing governer module looks exciting but not novel. Final results are not analyzed for statistical significance; those average results with STD do not estimate whether the proposed approach was worth the effort. This is an interesting incremental improvement of the previously published approaches to the not available online SEED dataset. The authors shall be encouraged to continue their research with possible end-to-end feature extraction, proposed DA, and final classification with statistical significance comparison. A precise BCI application shall also be outlined. It is hard to imagine a user switching emotional/affective states for an intentional control of a computer or machine. 

The paper proposes an affective emotion recognition method using EEG data with an aim to perform in the subject independent paradigm. Towards this, domain adaptation and generalisation methods are evaluated and in particular a pseudo domain adaptation method is proposed for the said purpose. Experiments are performed on SEED dataset, and a method that use pseudo domain adaptation and meta learning is proposed to improve upon computational and memory requirements in realising practical aBCIs.   Strengths
This research is targeted towards a very important and interesting area where emotions can be reliably detected for affective Brain Computer Interface. One of the major challenge of data drift arising from the non stationary nature of the EEG signal is theoretically addressed, as well as the practical implication are covered. 

Weakness
The authors should also review literature where the non-stationary nature of the EEG signal has been handled using appropriate feature selection. It is important to note that emotion recognition is also dependent on the task and authors need to add more justification to why each subject should be considered an independent domain when the same task is used to induce a particular emotion. 

Figure 2 needs to updated to represent the various action presented in Section 4.1

A general challenge in emotion recognition methods is the low accuracy when the number of emotions classes are increased. The authors have used three emotions, and based on there claimed novelty it would be important to see how the methods performs for lets say five emotions. 

The authors also need to add quantitative justification for comparing the computational and memory requirements of their proposed method, since this is one of the most important improvement expected when using this proposed method. 

Minor Correction:
Thoroughly read for structural and grammatical errors. 
Abstract- Correct aBIC to aBCI
Rephrase- aBCIs have received considerable attention very recent
 The authors have presented a method that covers the theoretical basis for emotion recognition in the presence of domain shift in data. However, existing methods use feature engineering to overcome the domain shift in EEG data which is inherent due to the non-stationary nature. Since the authors propose a domain adaptation method, to further validate the results and one way could be to test the performance using cross-dataset validation (such as DEAP). 

A new paradigm is called Pseudo Domain Adaptation (PDA) is proposed for subject-independent EEG-based affective computing. The method performs the pseudo-domain adaptation via meta-learning based on PDA and uses a feature extractor, a classifier, and a domain shift governor.  The results show that the proposed method doesn't require the additional computation resources of DA methods but achieves similar generalization performances. Following are the strengths and weaknesses in my opinion:

Strengths:
- Interesting problem. 
- Good literature review.
- Relatively good results.

Weaknesses:
- A number of typos and editing issues need to be corrected, e.g., page 4: "An continues map", page 5: "discrepancy. between", and many more.
- The description of the method is not very clear. Especially Section 4.2. does not clearly describe the method and many details are missing both about the method and motivation.
- The results, while strong, are not convincing. They are below the WGANDA 2018 (DA) and barely better than PPDA-NC 2021 and DResNet 2019 (DG). The SD is also larger than all three methods. 
- The experiments have only been performed on SEED. Only 1 dataset is not really sufficient to gauge the generalization to other datasets. -

The authors propose a PDA paradigm, in which no source domain is required in the test phase, for building subject-independent EEG-based affective model. Experimental results on the SEED dataset demonstrate that PDAML outperforms the DG methods and converges quickly. strengths:
1. The proposed model can achieve good performance by using only target domain data during testing.
2. The design of the sum-decomposable network is interesting.
3. I appreciate the subject-independent experimental setup.

weaknesses:
1. The experimental settings and data processing of the proposed method and the baseline methods are different, which does not seem to be a fair comparison. For example, the proposed method uses 15s EEG as a sample. But the WGANDA method (Luo et al.) uses 1s EEG as a sample.
2. I am worried about the reproducibility of the paper. I have read and run the code carefully. The seed set in each experiment is not exactly the same. In addition, the training accuracy of the pre-training process is difficult to reach 80%. And the loss function of meta-learning is difficult to converge during training. 
3. The author should compare the calculation amount and training time between the proposed model and the baseline methods. In the process of running the code, I found that it takes a lot of time to propose a model, especially meta-learning training.
4. The paper only uses one dataset. It is recommended to add more datasets. As far as I know, there are multiple data subsets in SEED, such as SEED IV. In addition, the evaluation index is also only acc. It is recommended to add multiple evaluation indicators.
5. Most of the baseline methods used by the authors are not recent methods. It is recommended that the author compare with the latest baseline method, such as [1-3]. And most of the more than 20 related works mentioned in related work are papers published before 2020. I suspect that the authors may have limited grasp of recent work in the field. Or the classification performance of the proposed method is not the current SOTA performance.
6. Different meta-learning strategies are used to train the governor and the complete network. There is no explanation as to why a different strategy was used.
7. Figure 4 lacks a detailed description of the pictures and legends.

[1] SparseDGCNN: Recognizing Emotion from Multichannel EEG Signals,IEEE TAC 2021
[2] SST-EmotionNet: Spatial-Spectral-Temporal based Attention 3D Dense Network for EEG Emotion Recognition,ACM MM 2020
[3] SFE-Net: EEG-based Emotion Recognition with Symmetrical Spatial Feature Extraction, ACM MM 2021


 This is an interesting work and research motivation is also valuable. But I am concerned about data processing, reproducibility, and model performance and evaluation.

Meta-review: The experimental part of the work has been reported by all reviewers as too limited and not convincing enough.
At this point this work cannot be endorsed for publication at ICLR.

Decision: Reject