MAtt: A Manifold Attention Network for EEG Decoding
Abstract
Recognition of electroencephalographic (EEG) signals highly affect the efficiencyof non-invasive brain-computer interfaces (BCIs).
While recent advances of deeplearning (DL)-based EEG decoders offer improved performances, the developmentof geometric learning (GL) has attracted much attention for offering exceptionalrobustness in decoding noisy EEG data.
However, there is a lack of studies on themerged use of deep neural networks (DNNs) and geometric learning for EEG decoding.
We herein propose a manifold attention network (mAtt), a novel geometricdeep learning (GDL)-based model, featuring a manifold attention mechanism thatcharacterizes spatiotemporal representations of EEG data fully on a Riemanniansymmetric positive definite (SPD) manifold.
The evaluation of the proposed mAtton both time-synchronous and -asyncronous EEG datasets suggests its superiorityover other leading DL methods for general EEG decoding.
Furthermore, analysisof model interpretation reveals the capability of mAtt in capturing informative EEGfeatures and handling the non-stationarity of brain dynamics.
Introduction and related works
A brain-computer interface (BCI) is a type of human-machine interaction that bridges a pathway frombrain to external devices.
Electroencephalogram (EEG), a non-invasive neuromonitoring modalitywith high portability and affordability, has been widely used to explore practical applications of BCIin the real world [1, 2, 3].
For instance, disabled users can type messages through an EEG-based BCIthat recognizes the steady-state visual evoked potential (SSVEP) induced by flickering visual targetspresented on a screen [4, 5, 6].
Stroke patients who need restoration of motor function undergomotor-imagery (MI) BCI-controlled rehabilitation as an active training [7, 8].
Most EEG-based BCIsystems are designed to detect/recognize reproducible time-asynchronous or time-synchronous EEGpatterns of interest, depending on the schemes of BCI [9].
For example, the MI EEG pattern is anendogenous oscillatory perturbation without an explicit onset time sources from the motor cortex[10].
On the other hand, a time-synchronous EEG pattern is time-locked to a specific event.
Forexample, the pattern of SSVEP is synchronized to the change of brightness on a flickering visualtarget.
The efficiency of BCI systems largely relies on the accuracy and robustness of the EEGdecoder.
However, due to the low signal-to-noise ratio (SNR) [11] and non-stationarity [12] of EEG,translating perplexing EEG signals into meaningful information has been a grand technology andscientific challenge in the field.
Recent advances in deep learning (DL) have contributed to the rapid development of DL-based EEGdecoding techniques [13].
DL models are capable of extracting features automatically according to given training data.
Convolutional neural network (CNN) is one type of the most common DL modelsand has achieved remarkable performance in tasks such as image recognition and object detection[14, 15, 16].
CNN models newly designed for EEG decoding use convolutional kernels that serveas spatial and temporal filters but with extra flexibility to optimize the transformation of EEG dataautomatically through model training [17, 18, 19].
In addition to the fast growth of DL-based EEGdecoders, geometric learning (GL) approaches, mostly based on Riemannian geometry (RG), havebeen adopted in the field of BCI [20].
RG is a type of non-Euclidean geometry that has a differentinterpretation of Euclid’s fifth postulate (i.e. parallel postulate) [21].
In GL, geodesic between pointson the manifold is a critical feature for classification tasks in BCI.
The power and spatial distributionof a segment of multi-channel EEG signals can be coded into a covariance matrix that is symmetricpositive definite (SPD) in general.
This allows mapping of EEG data directly onto a Riemannianmanifold where Riemannian metric is insensitive to extreme outliers and noise [22, 20].
In 2010,
Barachant et al.
[23] proposed Minimum Distance to Mean (MDM) that maps target EEG dataonto the SPD manifold to find the nearest class center.
Later on, they developed TSLDA [24] thatprojects data from the manifold to a specific tangent space where Euclidean classifiers are applicable.
RG-based classification for EEG decoding has shown extra robustness as the relationship betweendata samples can be stably preserved, leading to success in recent data competitions in the BCI fieldsuch as ’DecMEG2014’ 1 and the ’BCI challenge’ 2 .
The nascent field of geometric deep learning (GDL) has expanded by emerging techniques togeneralize the use of deep neural networks to non-Euclidean spaces.
Efforts have been madeto transitioning useful operations from Euclidean to Riemannian spaces, including convolution[25, 26, 27], activation function [25, 26], batch normalization [28, 29], that facilitate the ongoingdevelopment of GDL tools.
SPDNet [25] is a Riemannian network for non-linear SPD-based learningon Riemannian manifolds using bi-linear mapping that mimics Euclidean convolution for visualclassification tasks.
ManifoldNet [26] offers high performance in medical image classification withmanifold autoencoder.
[30] characterizes 3D movement via the manifold polar coordinate witha geodesic CNN.
[31] performs convolution on the manifold as a generalization of local graphor manifold pseudo-coordinate for vertex classification on graph and shape correspondence task.
In contrast of the vast develop of geometric deep learning (GDL) in many other scientific fields,only few studies focus on decoding EEG data with a merge use of GL and DL.
[32] proposed anetwork architecture that integrates fusion of Euclidean-based module and manifold-based modulewith multiple LSTM and attention structures to extract spatiotemporal information of EEG.
[33]proposes a Riemannian-embedding-banks method that separates the entire embeddings into multiplesub-problems for learning spatial patterns of MI EEG signals based on the features extracted fromthe SPDNet.
[34] combines federated learning and transfer learning on Riemannian manifold usingthe spatial information of EEG.
[35] proposes deep optimal transport on the manifold to minimizethe cost of domain adaptation from the source domain to the target domain.
[36] extracts multi-viewrepresentations of EEG.
These studies have established cornerstones toward the field of future GDLfor EEG decoding, but the increment of performance is yet marginal.
Most of the above-mentionedtechniques can not map the temporal information of EEG onto the manifold, or still rely on Euclideantools to handle EEG features.
We herein propose a manifold attention network, a novel GDLframework, which maps EEG features on a Riemannian SPD manifold where the spatiotemporal
EEG patterns are fully characterized.
The main contributions of the present study are the following:
• a manifold attention network proposed for decoding general types of EEG data.
• a lightweight, interpretable, and efficient GDL framework that is capable of capturing
spatiotemporal EEG features across Euclidean and Riemannian spaces.
• an empirical validation of our proposed model demonstrating its generalizable superiority
over leading DL approaches in EEG decoding.
• neuroscientific insights interpreted by the model that not only echo prior knowledge but also
offer a new look into the dynamical brain.
This article is organized as follows: we first brief the essential background of RG and manifoldattention mechanism; next, we describe the proposed mAtt architecture with details of model designand training; we then validate our proposed model experimentally; lastly, we interpret our proposedmodel with neuroscientific insights.
Preliminary
A manifold is considered as the expansion of curve and surface in Euclidean space.
It is a topologicalspace that can locally regarded as an open set in Hilbert space.
Suppose a manifold is endowed witha differential structure (i.e. a collection of charts satisfying transition mapping, which is definedon the overlap of charts), it is then the so-called differential manifold [37].
Riemannian geometryis a differential manifold equipped with Riemannian metric.
We consider the Symmetric positivedefinite (SPD) manifold, which allows us to manipulate manifold-valued data on the manifold directly.
The spatial information of EEG signal can be represented as a specific covariance matrix, whichrecords the relationship between channels, and is a critical index for us to understand EEG signal.
However, the solution of the Riemannian mean doesn’t have a close form, thus we need to calculatethe approximate mean in an iteration manner [23, 26] until convergence conditions are satisfied.
Riemannian mean is not suitable for being applied in deep learning because of the high computationalcomplexity.
Therefore, we have the Log-Euclidean metric below.
2.1 Notations
GL ( n, R ) :
= { A ∈ R n × n | determinant ( A ) ̸ = 0 } is a general linear group, which is the set of allreal non-singular sqaure matrices.
( M , g ) denotes connected Riemannian manifold.
Sym ( n ) :=
{ S ∈ M n × n ( R ) | S T = S } is the space of all n × n real symmetric matrices, where M n × n ( R )specifies the space of all real square matrices, ( . )
T is the transpose operator, and Sym + ( n ) :
={ P ∈ M n × n ( R )
| P = P T , v T Pv > 0 , ∀ v ∈ R n − { 0 }} is the set of all n × n symmetric positivedefinite(SPD) matrices.
<
A, B > F means the Frobenius inner product, defined as Tr ( A T B ) , where
Tr ( . ) is the trace operator.
Log ( . ) and Exp ( . ) are the principle logrithm operator for SPD matrix[38]and exponential operator for symmetric matrix respectively.
Both of them can be computed usingthe orthogonal diagonalization .
Log :
Sym + ( n ) (cid:55)→
Sym ( n ) is an operator that maps a SPD matrix
P ∈
Sym + ( n ) to Sym ( n ) by:
[Equation]
where U is the matrix of eigenvectors of P , since P ∈
Sym + ( n ) , σ i > 0 , i = 1 , ..., n
The inverse projection is Exp of symmetric matrix:
Exp : Sym ( n ) (cid:55)→
Sym + ( n ) , an operator mapsa symmetric matrix S ∈
Sym ( n ) to Sym + ( n ) by:
[Equation]
where V is the matrix of eigenvectors of S .
2.2 Log-Euclidean metric
Log-Euclidean metric (LEM) offers a more simple, similar, and efficient generalization to calculatethe center on the SPD manifold than Affine-invariant metric (AIM) [39, 40].
LEM is a bi-invariantmetric on the Lie group on the SPD manifold [40].
The geodesic distance from P 1 to P 2 on the
Sym + ( n ) is also given by [40]:
[Equation]
Furthermore, we can also define the Log-Euclidean mean( G ) via the Log-Euclidean distance:
[Equation]
[Figure]
Fortunately, the solution to the formula above has a closed form to follow, given by [41]:
[Equation]
[Figure]
In this work, we are going to use the weighted Log-Euclidean mean that is endowed with differentweights in different P l .
We denote the weight of each P l as w l , where ∀ l ∈ { 1 , 2 , ..., k } .
Here,{ w l } kl =1 satisfies the convexity constraint definition (i.e. k (cid:88) l = w l = 1 , and w l > 0 ).
The definitionand the corresponding weighted Log-Euclidean mean can be defined and derived as:
[Equation]
and
[Equation]
=
[Equation]
respectively.
Methodology
As shown in Figure 1(a), the architecture of mAtt includes components of the feature extraction(FE), the manifold attention module, transitioning from Euclidean to Riemannian space (E2R), andtransitioning from Riemannian to Euclidean space (R2E).
3.1 Feature extraction of EEG signals
We use two convolutional layers to extract information of the raw EEG signals, where the firstconvolutional layer performs spatial filtering to the multi-channel EEG signals and the secondconvolutional layer extracts spatiotemporal features.
Our parameter setting follows [19].
3.2 From Euclidean space to SPD manifold(E2R operation)
As illustrated in Figure1 (b), we convert the embeddings from the feature extraction stage to the SPDdata and map the feature embeddings from Euclidean space to the SPD manifold.
Suppose ˜ f denotesthe embeddings after the feature extraction stage, we divide the whole embeddings into several epochs˜ f 1 , ˜ f 2 , ..., ˜ f m , and calculate the sample covariance matrix(SCM) of each ˜ f
i , ∀ i ∈ { 1 , 2 , ..., m } .
Bydoing so, we get a sequence of covariance matrices that present the temporal information of theembeddings ˜
f
in the form of SPD data, called SCM ˜ f 1 , SCM ˜ f 2 , ..., SCM ˜ f m .
After we get somedatapoints, we do trace-normalization and add a small number ϵ on each main diagonal element for
[Figure]
each SCM ˜
[Equation]
i (i.
[Equation]
tr (cid:16) SCM ˜ fi (cid:17) + ϵI ) where i ∈ { 1 , 2 , ..., m } , I is the identity matrix, and we set ϵ as1e-5 in our source code.
The resulting SPD sequence is denoted as ˜ X =
[ ˜ x 1 , ˜ x 2 , ..., ˜ x m ] .
We add asmall identity matrix on them to promise ˜ x i to be a well-defined SPD matrix.
3.3 Manifold attention module
The input of this layer is a sequence of SPD data.
The overview of the manifold attention module isillustrated in Figure 2 (a).
Motivated by [25] and [42], we capture the spatiotemporal information onthe manifold.
Suppose the module takes a sequence of SPD matrices [ ˜ x 1 , ˜ x 2 , ..., ˜ x m ] , denoted as ˜ X .
Here we have the query, key, and value in the form of SPD matrices on the manifold [42].
We convertthe ˜ x
i to the q i , k i , and v i via bilinear mapping and exploit non-linear and valid features from eachsegment.
Suppose the shape of ˜ x
i is d c × d c , and h q , h k , and h v is the mapping from ˜ x
i to q i , k i ,and v i respectively:
[Equation]
where ˜ x i ∈
Sym + ( d c ) , W q , W k , and W v ∈ R d u × d c ( d u < d c ) denotes transformation matrices.
Tomake sure the output q
i , k i , and v i are also SPD matrices, transition matrices W q , W k , and W v arerow-full rank matrices.
After we got q i , k i , and v i by bilinear mapping, we define the similarity for measuring the q i and k j
SPD matrices.
In Euclidean space, there are several ways to define the similarity.
A most commonway is to use dot-product[42] to measure the similarity of query and key.
However, our query, key,and value are SPD matrices instead of vectors as regular attention.
We define the similarity basedon the Log-Euclidean distance (equation(2)) between query and key.
Suppose we have q i and k j ,for some i, j ∈ { 1 , ..., m } .
The similarity sim ( . ) is a strictly decreasing function of distance[0 , ∞ )
(cid:55)→ [0 , 1] and is defined as: sim ( q i , k j ) =
[Equation]
(1+
[Equation]
L ( q i ,k j )) :
= α ij .
Then, the attentionmatrix is:
[Equation]
We then use Softmax function to shrink the range along the row direction, making values in rowhave convexity constraint property.
The final attention probability matrix A ′ is:
[Equation]
where α ′ ij = exp ( α ij ) (cid:80) mk =1 exp ( α ik ) , ∀ i, j ∈ 1 , · · · , m .
Finally, we combine the attention probability matrixand v 1 , v 2 , ..., v m to get the final output v ′ 1 , v ′ 2 , ..., v ′ m and define the output v ′
i ( ∀ i = 1 , 2 , ..., m ) via
Log-Euclidean mean as:
[Equation]
The whole mAtt procedure is illustarted in Algorithm 1 in appendix.
As shown in Figure2(b), the output v ′ 1 of our attention module can be comprehended as a projectionthat translates the weighted sum, on the tangent space, of three different matrices encoded bythree different epochs and corresponding attention scores α ′ 11 , α ′ 12 , α ′ 13 (or weights) to a specificrepresentative matrix v ′ 1 on SPD manifold.
Herein the weights for the weighted sum on the tangentspace is assigned by its query matrix q 1 and corresponding keys k 1 , k 2 , k 3 to generate the relevancescore between q 1 and k 1 , k 2 , k 3 [42, 43].
3.4 From Riemannian manifold to Euclidean space(R2E) and loss layers
After passing through attention module, ReEig layer is used to imitate the ReLU function.
But it isdifferent from the ReLU function that sets the threshold to the value of input, ReEig sets a thresholdto the eigenvalue of the input, can be defined in [25].
R2E operation aims to map the SPD databack to the Euclidean space for executing the final classification, which is composed of a Log layerand regular flatten layer in [25] sequentially.
Log layer is the most common skill in geometric deeplearning to project the SPD data to the Euclidean space.
By doing so, we reduce the manifold to a flatspace by Log ( . ) operation.
We take Log ( . ) operation on the output from the attention module layerv ′ 1 , v ′ 2 , ..., v ′ m ∈
Sym + ( d u ) .
Denote the whole R2E operation as h L :
Sym + ( d u ) (cid:55)→ R d u × ( d u +1) / 2 :h L ( v ′ i )
= flatten ( Log ( v ′ i ))
= flatten (cid:0) S ( diag ( log ( σ 1 ) , · · · , log ( σ d u )))
S T (cid:1)where S is the eigenvector-matrix of v
′ i , and σ 1 , · · · , σ d u are the eigenvalues of v
i .
The Log ( . )operation is the same as equation(1), and the flatten ( A ) operation flatten the upper triangle of thearbitrary symmetric matrix A .
Finally, we set a fully connected layer and regular softmax operation on embeddings after R2Eoperation.
Suppose the output from the whole model stream is ˆ y, the groundtruth is y, we define theloss L as the cross-entropy loss of ˆ y and y .
Experiments
Here we evaluate the proposed mAtt using both time-asynchronous and time-synchronous EEGdata to give empirical evidence of the advantages.
The performance in a general use for EEGdecoding is compared against leading DL-based techniques.
We incorporate the BCI Competition
IV 2a Dataset (BCIC-IV-2a) [44] to assess the performance on time-asynchronous motor-imagery(MI) EEG decoding , the MAMEM EEG SSVEP Dataset II (MAMEM-SSVEP-II) [ ? ] and the
BCI challenge error-related negativity (ERN) dataset (BCI-ERN)
[45] to assess the performance ontime-synchronous SSVEP and ERN EEG decoding.
Previous and current state-of-the-art DL-basedmodels listed for comparison with mAtt include MBEEGSE [46], TCNet-Fusion [47], EEG-TCNet[48], FBCNet[49], SCCNet
A series of experiments were conducted to evaluate the performances of the mAtt against other
EEG decoders with the context of real-world BCI usage taken into account.
In the real-world usageof BCI, a user usually needs to go through a training session for collecting a sufficient amount ofindividual EEG data for training the decoding model before executing the BCI system.
To stick withthe practical scenario, we performed an individual training scheme where a chunk of trials within asubject are assigned to the training set and the left-over trials within the same subject are used fortesting [19, 49].
For the BCIC-IV-2a dataset, we used the first session of a subject to the training set where one out of eight was used for validation for mAtt with m = 3 .
The model with the lowestvalidation loss within 350 iterations was used for testing on the second session of the same subject.
For the MAMEM-SSVEP-II/BCI-ERN dataset, we assigned the first four sessions of a subject to thetraining set where one out of four was used for validation for mAtt with m = 7 / m = 3 .
The modelwith the lowest validation loss within 180/130 iterations was used for testing on the fifth session ofthe same subject.
The classification performances for BCIC-IV-2a and MAMEM-SSVEP-II datasetswere estimated by the mean accuracy across ten repeats for each subject.
On the other hand, weuse the same criterion as [17], herein the AUC score [50] is adopted to estimate the performance of
BCI-ERN dataset due to the imbalanced issue.
4.1 Performance comparison
We validate the performance of mAtt against other leading methods.
The criteria of selecting thebaseline model collection here is based on: 1) code availability and completeness and 2) solidevaluation (e.g. cross-session) without additional auxiliary procedures (e.g. manual feature extraction,data augmentation, pre-trained model, etc).
As shown in Table 1, mAtt outperforms all other leading
DL methods on both time-synchronous (SSVEP) and -asynchronous (MI) EEG decoding.
However,for ERN dataset, mAtt won the second place, only 1% lower than the EEG-TCNet but at least 7%higher than EEG-TCNet on other two kinds of EEG decoding tasks.
Nowadays a variety of DLmethods employed in EEG classification focus on a specific type of EEG decoding task due to highvariability between different types of EEG data [49, 48, 19].
It is difficult to design a DL architecturefor decoding various type of EEG data.
The table attests the robustness of proposed mAtt, whichhas strong generalization capacity to adapt general types of EEG data compared to other leading
DL models.
There are about 3% leap forward the best baseline models on both MI and SSVEP task.
The rigorous cross-session training scheme and the preprocessing step that will highly decrease thetime resolution of each EEG data we adopted to validate the performance for each model and datasetmaybe a reason why some of baseline models may not decode EEG dynamics successfully.
Moreover,the rarity of the EEG data may cause arduous overfitting issue for baseline models.
We conclude thatmAtt has a generalizable superiority in decoding EEG for various types of BCI systems.
4.2 Ablation study
We assess the significance of each of the major component in mAtt via a series of ablation analysis.
As shown in Table 2, the accuracy reduces if we only use one component in our mAtt to do theclassification.
However, the combination of our proposed manifold attention module and featureextractor (FE+MA) achieves the best accuracy on all datasets.
This implies that there is no redundantcomponent in our proposed mAtt, and each part is needed for its non-negligible functions.
Thefeature extractor aims to denoise and preprocess the EEG signals, and the attention module focus onintegrating the preprocessed EEG signals and capturing underlying dynamics in the latent features.
[Table]
[Table]
We can further compare the result between FE+SA and FE+MA to check the necessity of MA:the performance of FE+MA significantly outperform the regular self-attention based FE+SA on allvalidation EEG datasets.
4.3 Model interpretation
Through analysis for the interpretation of the proposed model, mAtt, we are able to uncover theunderlying characteristics learnt from the data.
Figure 3 illustrates the gradient response for MI EEGdecoding across channel and across time.
We can see left/right hand MI responses are strong at
C4 and C3 corresponding to right/left motor cortices that control the lateral motor functions of thecontralateral side of the body [53].
Both feet and tongue MI, that are not lateral movements, presentsstrong responses at CPz above the midline of motor cortex.
The spatial distribution clearly exhibit
[Figure]
[Figure]
asymmetric pattern for left/right hand MI and symmetric pattern for feet/tongue MI.
The temporalinformation is available in Figure 4, where all four types of MI induce strong response at mu bandaround 10 Hz.
This result is in line with the well-known association between motor function andmu rhythm in EEG recordings [53].
The visualization of our model for SSVEP decoding is exhibitin the appendix.
Figure 5 depicts the distribution of attention scores across epochs between the (a)
MI, (b) SSVEP, and (c) ERN EEG signals.
Here, the attention score refers to the average of therelevance score of an attention network, as described in [42], and has been applied to interpretingan attention-based EEG decoder [43].
For MI EEG signals, attention score is the highest at the firstepoch and decreases in the following epochs, which implies that the beginning of the motor imagerymay contribute a higher importance determined by the manifold attention module.
The profile ofattention score for SSVEP EEG signals presents a similar traits that the earlier epochs relate to higherimportance.
For ERN EEG signals, attention score is the highest on the first two epochs.
As weobserve a consistency cross EEG datasets that higher attention scores lie in earlier epochs, this mayinfers that the attention module relies largely on the similarity to the early stage of a trial, which isanalogous to baseline correction, a major common procedure in conventional EEG signal processing[54].
This analysis reveals the capability of mAtt in handling the non-stationarity of the dynamicalbrain activity.
4.4 Limitation
In our framework, vacuum permittivity ϵ is added on all main diagonal elements of covariance
cx
i x
Tito ensure the rigor of SPD matrix.
But the operation may cause the repeated singular value ϵ in S i .
Therefore, we proposed possible solutions for this issue: 1) Let m < n when dividing the embeddingsinto several time segments, reducing the possibility of getting low-rank S i ; 2) Let ϵ be randomlydrawn from a specific distribution, such as Uniform (1 e − 8 , 1 e − 4) to solve this issue, which isalso a practicable solution; 3) Use the derivative of a low-rank matrix[55] to cope with this issue.
Conclusion
We propose a manifold attention network as a novel GDL framework for decoding both timesynchronous and -asynchronous EEG decoding.
Using back propagation based on the Stiefelmanifold, the proposed mAtt is capable of mapping EEG features onto a Riemannian manifold,where spatiotemporal EEG patterns are captured and characterized, within a lightweight architecture.
The experimental results suggest the superiority of mAtt over current leading DL methods for bothtime-synchronous and -asynchronous EEG decoding.
With the interpretability of mAtt, we visualizethe spatial and temporal EEG patterns, which are in line with prior neuroscientific knowledge andshed light on potential possibility of tracking the brain dynamics.
In sum, our privileged method,mAtt, improves the SOTA performance of EEG decoding, and is expected to impact on GDL-based
EEG processing with generalizable efficiency and robustness for future development of various BCIsystems.
Checklist
Since we provided a wrong checklist for reviewers, we offer a new checklist here for reviewers tocheck.
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] We elaborated the contribution and scope of present
study in Section1.
(b) Did you describe the limitations of your work?
[Yes] We discussed not only limitation,
but also the possible solutions for our limitation in Section4.4.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section3and appendix.(b)
Did you include complete proofs of all theoretical results?
[Yes] See Section3 andappendix.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] We provide theinstructions and code in the supplemental material for everyone can reproduce
our result easily.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] We describe the training scheme in Experiments, and the
detailed parameters are available in the source codes.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] We repeat ten times for any experiment with randominitialization and report the standard error for each cell.
(d) Did you include the total amount of compute and the type of resources used (e.g.,type of GPUs, internal cluster, or cloud provider)?
[Yes] We elaborate on this in the
appendix.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We adequately cite
any existing assets used in this work.
(b) Did you mention the license of the assets?
[Yes] It is provided in the supplemental
materials.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]