Title: Learning Cortico-Muscular Dependence through Orthonormal Decomposition of Density Ratios
Venue: NeurIPS 2024
Accepted: yes
Keywords: EEG-EMG fusion

statistical dependence

orthonormal decomposition

cortico-muscular connectivity
Abstract: The cortico-spinal neural pathway is fundamental for motor control and movement execution, and in humans it is typically studied using concurrent electroencephalography (EEG) and electromyography (EMG) recordings. However, current approaches for capturing high-level and contextual connectivity between these recordings have important limitations. Here, we present a novel application of statistical dependence estimators based on orthonormal decomposition of density ratios to model the relationship between cortical and muscle oscillations. Our method extends from traditional scalar-valued measures by learning eigenvalues, eigenfunctions, and projection spaces of density ratios from realizations of the signal, addressing the interpretability, scalability, and local temporal dependence of cortico-muscular connectivity. We experimentally demonstrate that eigenfunctions learned from cortico-muscular connectivity can accurately classify movements and subjects. Moreover, they reveal channel and temporal dependencies that confirm the activation of specific EEG channels during movement.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a novel approach called Functional Maximal Correlation Algorithm with Trace cost (FMCA-T) for estimating cortico-muscular dependence by leveraging orthonormal decomposition of density ratios. This method is designed to model the relationship between EEG (electroencephalography) and EMG (electromyography) signals, addressing the challenges of interpretability, scalability, and local temporal dependence in cortico-muscular connectivity. The key contributions include introducing a matrix trace cost optimization for improved stability and efficiency, demonstrating robustness against nonstationary noise and delays, and effectively capturing movement and subject information from EEG features for enhanced classification accuracy. The proposed method outperforms existing baselines, particularly in cross-subject scenarios, and provides insights into channel-level and temporal dependencies, reinforcing its potential applications in brain-computer interface development and neuromuscular disorder diagnostics.\n\nSTRENGTHS:\n1. Innovative Method: Introduces the Functional Maximal Correlation Algorithm with Trace cost (FMCA-T), providing a novel approach for estimating cortico-muscular dependence.\n2. Improved Stability and Efficiency: Utilizes matrix trace cost optimization, which is more stable and computationally efficient compared to traditional log-determinant cost methods.\n3. Enhanced Classification Accuracy: Effectively captures movement and subject information from EEG features, significantly improving classification accuracy, especially in cross-subject scenarios\t\n4. Validation on Multiple Datasets: Validated using both simulated and real EEG-EMG datasets, confirming the method’s effectiveness and robustness.\n5. Open Data and Reproducibility: Offers open access to datasets and detailed implementation code, facilitating reproducibility and further research in the field.\n\nWEAKNESSES:\nThe provided baselines are relatively few; future work could expand on this.\n\nQUESTIONS:\n1. While the paper discusses the improved stability and efficiency of the FMCA-T method, can the authors provide more detailed about the computational resources required for training and inference?\n\nLIMITATIONS:\n1.Limited Dataset Size: The cross-subject classification performance drops, potentially due to the limited dataset size of only 25 participants. Larger and more diverse datasets may be needed to validate the method’s robustness comprehensively.\n2. Generalization to Other Modalities: While the method shows promise for EEG and EMG signals, its generalizability to other types of biosignals or broader neural data modalities has not been extensively disscuss.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents a new method to model the relationship between cortical and muscular oscillations using EEG and EMG recordings. Traditional methods like Cortico-Muscular Coherence (CMC) have limitations, so the authors propose using statistical dependence estimators to learn eigenvalues, eigenfunctions, and projection spaces. This approach improves interpretability, scalability, and local temporal dependence. Experimental results show that the method accurately classifies movements and subjects, highlighting specific EEG channel activations during movements, and demonstrates robustness against noise and delays, suggesting its potential for diagnosing neuromuscular disorders and developing brain-computer interfaces.\n\nSTRENGTHS:\n1. The paper combines statistical dependence estimators with neural network optimization techniques. This fusion of methodologies enhances the ability to capture high-level and contextual connectivity between cortical and muscular oscillations.\n\n2. The paper provides a detailed description of the proposed methodology, including the mathematical foundations, algorithmic implementation, and practical considerations. The inclusion of eigenvalues, eigenfunctions, and projection spaces adds depth to the analysis.\n\n3. The authors conduct comprehensive experiments to validate their method. The results demonstrate the method's robustness against nonstationary noise and random delays, confirming its reliability and practical applicability.\n\nWEAKNESSES:\n1. Mathematical and Algorithmic Complexity: The proposed method involves complex mathematical formulations and advanced statistical techniques that may be challenging for a broader audience to grasp. Simplifying some of the mathematical derivations or providing more intuitive explanations and visualizations could make the paper more accessible.\n\n2. Interpretation of Results: While the method highlights specific EEG channel activations during movements, the physiological and neuroscientific significance of these results could be further elaborated. Providing more detailed discussions on how these findings align with or differ from existing neuroscience research would enhance the interpretability and relevance of the results.\n\n3. Scalability: The scalability of the proposed method to larger datasets or longer signal durations is not thoroughly addressed. Discussing the computational complexity and providing benchmarks on how the method performs with varying data sizes would be valuable.\n\nQUESTIONS:\n1. The method identifies specific EEG channel activations during movements. Could you provide more detailed explanations or references to how these findings align with existing neuroscientific knowledge? What are the physiological implications of the identified activations, and how do they contribute to our understanding of cortico-muscular connectivity?\n\n2. The author mention potential applications in diagnosing neuromuscular disorders and developing brain-computer interfaces. Can you provide concrete examples or case studies where your method has been or could be applied? What specific benefits or improvements does your method offer over existing approaches in these applications?\n\nLIMITATIONS:\nPlz go and check weaknesses and questions.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors apply novel but already existing (https://www.sciencedirect.com/science/article/pii/S0047259X2300074X, https://arxiv.org/pdf/2212.04631) machinery based approach on the orthonormal decomposition of density to decipher the relationship between cortical brain activity and the electromyographic signals during basic hand movements. The work is based on the publicly available dataset and the code is available. The unknown decomposition is modeled by a pair of neural networks concurrently processing EEG and EMG data in order to arrive at the internal representation for each of the modality. The internal representations are then aligned to minimize the rank of the joint covariance matrix. To guide the learning process, the authors propose a somewhat novel loss function equal to the negative trace of the canonical correlation matrix calculated using latent representations. The authors test their approach on the downstream tasks of classifying movement types in both within and across subject designs. They also apply the obtained representations to distinguish between participants based on their EEG data. The authors provide some interpretation to the obtained solution in the form of channel and temporal maps indicating the electrodes and time moments that contribute to the decoding most.\n\nSTRENGTHS:\n1. The authors applied novel but existing methodology of orthonormal density decomposition to the EEG+EMG dataset for the first time \n2. The authors introduced a novel loss function and showed that it provides better performance in the downstream task of EEG-based classification of movement types\n3. The authors used multi subject dataset \n4. The authors attempted to provide interpretation of the obtained decision rule\n5. The authors present detailed results of their experiments in the appendix\n\nWEAKNESSES:\n1. Several inaccuracies and lack of details in the mathematical expressions:\n1.1 line 100, last expression and additional p(z) is needed in the integral\n1.2 equation 3  - do the eigenvalues need to be normalized? Does the sum exclude the first normalized eigenvalue?\n2. I would argue against the suggested novelty of the proposed loss function as it seems like the loss function that is closely associated with the Canonical correlation analysis (CCA) (equation 4). Generally speaking, the proposed approach boils down to the CCA in the latent variable space with latents computed by means of a CNN.\n3. The authors claim that they “..design a specialized network topology to generate features for individual channels and time intervals, ensuring that the internal layers of this network quantify channel-level and temporal-level features, similar to [22–24]” - however unlike for instance the EEGnet, the authors use non-linearities in the temporal network (prior to the spatial) which in my view prevents the straightforward interpretation of the obtained representations at least using simple correlational measures. \nSee also Q.1 and 2. \n4. The authors did not validate their approach to interpreting the decision rule and obtaining spatial and temporal maps with simulated data. This needs to be done and the simulated data should contain not only the neuronal sources coupled to the simulated EMG but also the sources unrelated to the signal of interest (EMG). The authors then need to demonstrate that their methods infers the proper spatial patterns corresponding to the task-relevant simulated neuronal sources. Ideally,  the obtained maps should be ranked based on their importance for the overall decoding accuracy. If this is not possible within the review cycle, the authors should significantly reduce the proportion of the manuscript dedicated to the physiological plausibility of their solution and instead describe limitations related to potentially non-physiological origin of the extracted features.  \n5. It is disappointing that when interpreting the decision rule the authors did not provide information regarding the EEG frequency domain their network got tuned to during the training.\n\nQUESTIONS:\n1. Having significant experience in the domain of recording and analyzing electrophysiological data I founnd the obtained maps very suspicious. While EEG electrode FC1 can indeed be implicated and be coherent with EMG, I would expect other electrodes such as C3, C5 to have some significant contribution to the EEG derived latents that are maximally aligned with EMG. Instead in addition to FC1 we see the involvement of peripheral electrodes and the frontal electrodes. These electrodes often lose proper contact with the skin and become sensitive to the physical movements due to capacitive effects, when slight body displacements during the actual movement causes significant fluctuation in the electrode-skin capacitance and modulates the signals registered by EEG. The analysis of frequency response of the temporal layer (see W.5) may help to resolve this potential issue.\n\n2. In the dataset used by the authors the reference channel was located in the midline between FC1 and FC2 sensors. Such an arrangement often results in low variance of the signals located close to the reference. The spatial patterns that the authors demonstrate in Figures 5 and 11 show peaks around Fc1 and FC2. The ground electrode was located at the edge of the EEG cap between F1 and F2 and the temporal maps show them as the next best electrodes after Fc1. Could it be that within certain normalization steps the authors explicitly or implicitly  divided the data by the channel variance or multiplied the data by a poorly conditioned and not properly regularized inverse covariance that the role of these electrodes got artificially inflated?\n\n3. The authors show spatial maps for several other *selective* subjects and clusters to illustrate across subject reproducibility. What about the spatial maps corresponding to the other latent channels\\clusters for SUB3? \n\n4. Why did the authors not follow the EEGNet architecture and decided to use non-linearities between the temporal and spatial processing blocks? Avoiding nonlinearities in the front end would improve interpretability and would help to make the presentation more convincing. \n\n5. How do the authors avoid the trivial training result, i.e. that the two networks will simply learn to generate similar EMG and EEG embeddings regardless of the input data?\n\nLIMITATIONS:\nThe authors partly addressed limitations but several items, see Weaknesses section, are left out.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach to analyzing cortico-muscular connectivity using statistical dependence measures based on density ratio decomposition. The authors apply a method called Functional Maximal Correlation Algorithm with Trace cost (FMCA-T) to paired EEG and EMG recordings. The key idea is to decompose the density ratio between EEG and EMG signals into eigenvalues and eigenfunctions, which can capture important contextual information that affects the EEG-EMG dependency such as type of movement or subject without having them labeled. They also use the learned eigenfunctions as feature projectors and train a classifier on top for movement type classification tasks.\n\nThe authors test their approach on simulated data (SinWav) and a real EEG-EMG dataset with 25 subjects performing 11 different upper limb movements. They compare FMCA-T against several baseline methods for dependence estimation and classification. They find that the learned eigenfunctions capture factors such as movement type and subject identity. Further, FMCA-T outperforms the baselines, for example by 10% for cross-subject classification of arm-reaching, hand-grasping and  wrist-twisting.\n\nSTRENGTHS:\n* very sophisticated method with clear motivation\n* original idea cleanly mathematically derived (as far as I can tell)\n* produces good results\n\nWEAKNESSES:\n* classification baselines, could be stronger, e.g. by also using [EEG Conformer](https://pubmed.ncbi.nlm.nih.gov/37015413/)  and [Deep4](https://onlinelibrary.wiley.com/doi/full/10.1002/hbm.23730)\n* text is very dense at times, definitely found some part hard to read, but not sure how much it can be made easier, possibly you could explain some concepts used in 2.2 in more detail in the supplementary\n\nQUESTIONS:\n\"In scenarios where X and Y are statistically  independent, all eigenvalues are zero\" > doesn't this make the density ratio 0 then? shouldn't the density ratio be 1 if they are independent?\n\nLIMITATIONS:\nAuthors could discuss a bit more under what conditions the assumption of conditional independence may be problematic and when it is fine for EEG/EMG. In terms of what one may expect to see in analyses as performed here under different conditions.\nOf course, evaluation on further datasets would also be helpful for this.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-16"}