Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: Inverse modeling is at the heart of medical imaging techniques, especially in neuroimaging. Many techniques have been developed in recent years, first based on PDE techniques and, more recently, by machine learning approaches. The addition of a priori knowledge to these techniques allows a better understanding of the studied phenomenon. Therefore, it is necessary to develop robust methods combining modeling, a priori knowledge, while controlling the computational cost. This article proposes to derive an efficient inference algorithm with spatio-temporal dynamics in model parameters and noise. The spatial and temporal duality is realized by using the product of 
Kronecker product. They assume sparse spatial covariance matrices, while the temporal covariance is modeled to have either full or Toeplitz structure. They then define a Bayesian model whose parametric estimation is performed by a Maximization-Minimization algorithm. The proposed results are motivated by demonstrations (detailed in appendix) and numerical experiments.
main_review: The article is well constructed, as are the appendices, which provide many insights.

The demonstrations, postponed to the appendix, are pleasant to follow. Moreover, for those I have reread, at least, they seem correct unless I am mistaken. However, they are essentially based on Riemannian geometry, which may reduce their scope. The authors have nevertheless made an effort to give the essential definitions in appendix C (even if I doubt that this is sufficient for a "neophyte" in this field).
I particularly appreciated that each theorem stated in the article is precisely referred to its proof.

The chosen approach seems innovative. However, I wonder what sense it makes to assume the same temporal correlation for noise and sources in terms of modeling. And to what extent is this assumption realistic, encountered in practice, reasonable? I am aware that this makes the computation easier, as claimed by the authors in the discussion. Are the authors convinced that this assumption can also be relaxed theoretically and not just empirically as is currently the case?

Small remarks: prefer to use a git repository rather than a folder on a dropbox, which seems to me less perennial.
Also, Figure 1 is not readable when printed in black and white: I suggest that authors use different symbols for each node of the curves and include them in the legend.
limitations_and_societal_impact: The article discusses methods to improve medical imaging techniques. Unless medicine is misused, it seems to me that it cannot have a negative social impact (but I could be wrong).
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes an efficient method for source reconstruction from M/EEG data that estimates the spatiotemporal correlation structure in both the source space and the measurement error. The spatiotemporal pattern of the source and noise covariances is given by a Kronecker product structure with an independent (diagonal) spatial structure and either a full or Toeplitz temporal structure, where the temporal component is shared across the source and noise components. Since the full optimization problem is non-convex, the authors propose a majorization-minimization (MM) algorithm with provable convergence and derive simple, efficient iterative updates for each component. Two versions of the model are proposed: full Dugh, which uses a full positive definite matrix for the temporal covariance; and thin Dugh, which can be applied in the stationary data setting and uses a Toeplitz temporal covariance structure with an alternative set of efficient updates for the MM algorithm. On simulated data with known sources, thin Dugh achieves comparable or better performance as compared to existing source reconstruction methods (Champagne, eLORETA, and S-FLEX). When applied to simulated data that was generated while varying the true underlying temporal covariance structure, full Dugh is shown to outperform thin Dugh when the true matrix is full and vice versa when it has Toeplitz structure. Finally, experiments on real MEG data from subjects presented with visual or auditory stimuli show that full and thin Dugh can recover plausible and well-localized temporal and spatial patterns for the reconstructed sources, even with a very limited number of trials.
main_review: Overall, the paper is clearly written. The presentation of the method is sound, there is enough technical detail to understand the model and algorithm, and the experiments do a sufficient job of demonstrating the Dugh method's utility both when compared to other source estimation approaches on simulated data and for recovering plausible localized activity on real MEG data. I particularly appreciated the comparison between thin Dugh and full Dugh, both on the real MEG data as well as on synthetic data with known structure to demonstrate that each version of the method excels when the underlying structure aligns with exactly what it was designed for.

One issue I had with the paper was the limited discussion of and comparison to other source imaging methods that account for temporal structure in some way. This could include approaches such as mixed-norm inverse methods, which do encourage some temporal smoothness in the estimated source activations even if they don't model a full temporal covariance structure as the proposed method does. While such other methods are briefly referred to in the paper, comparing to other approaches that model temporal structure could further showcase the particular strength of the Kronecker-factored covariance structure and associated optimization algorithm that the authors present.

Another interesting experiment would have been a runtime comparison between the proposed method and the other methods used in the experiments. The authors claim their algorithm is efficient and derive a set of simple closed-form updates, but no direct comparison of efficiency is made against the other source estimation methods. In particular, this would validate the point the authors make about other methods that model temporal structure having "computationally over-demanding inference schemes" (line 6). A runtime vs. accuracy comparison to other spatio-temporal methods as well as the baselines already included in the paper would strengthen the presentation of the proposed Dugh method as an accurate source estimation method that is also less computationally intensive compared to alternatives that also model temporal structure.

Other minor comments:
- The caption for Figure 2 mentions "Toeplitz structure in the first row, and full covariance structure in the second row", when the figure shows the reverse
limitations_and_societal_impact: The authors have discussed the technical limitations of their work, but not the potential negative societal impact. The most salient potential impact of this work is its use in neuroscience research, where any potential technical biases or shortcomings of the method could be reflected in the findings of any researcher who applies this method to their data. The authors could briefly discuss this potential impact in the final section of the paper. Beyond this, there are no obvious harmful societal consequences of this work to my knowledge.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This manuscript presents an efficient majorization-minimization optimization framework to infer the parameters of the spatio-temporal covariance matrix in the MEEG signal source reconstruction problem. The main contribution lies in simplifying a non-convex optimization problem using a convex surrogate function with a convergence guarantee. The method has been tested in few limited scenarios on simulated and real MEEG data and the results are compared against three alternative approaches. 

main_review: *** Main Comments ***
- This paper is addressing an interesting and significant problem in the neuroimaging context.
- The idea of using the specifications of the Kronecker product in disentangling the spatio-temporal structure of the covariance matrix is interesting, however, this idea is not original and was presented in previous studies for the purpose other than MEEG source reconstruction.
- The text is clear but dense in mathematical notations and representations.
- This work's significance is limited by the strong (restrictive and wrong) assumptions that are made on 1) the independence of the sources across different brain locations, 2) identical temporal distribution for signal and noise. The authors are aware of these practical limitations and conducted a simulated study in a limited scenario arguing their negligible effect on the outcoming results. 

*** Other Comments ***
- The authors categorize the problem in Eq. 1 as a multi-task regression problem, while originally the addressed problem diverges from the definition of multi-task learning in which the aim is to estimate $\mathbf{L}$ and not $\mathbf{X}$ (unlike brain source reconstruction).
- I would call $\Gamma$ as spatial and $\mathbf{B}$ as temporal covariance matrix because even though the $\Gamma$ is computed on time-series it at the end represents the spatial structure of the signal across different brain locations (and the same for $\mathbf{B}$).
- According to the ground-truth covariance matrix in Figure 2, the simulated signal can barely be considered structured. I suggest simulating data with a more salient structure in the signal and noise components to better evaluate the violation of assumptions.
- Despite considering the spatial structure of data, I wonder why the reconstructed sources are so sparse in Figure 3. I would expect a more smooth localized effect. 
- It is not clear to me in what sense the proposed model is hierarchical. In the Bayesian context, hierarchical models have a specific definition (using the same prior in a hierarchy over a group of parameters). 
limitations_and_societal_impact: The proposed method in this study suffers from strong unrealistic assumptions on the distribution of noise and signal. The authors are aware of these limitations and conducted a simulation study to investigate their effect. However, the conducted simulated study in not well-designed with respect to actual realistic scenarios. I suggest:

- to simulate data with a more salient spatio-temporal structure (the covariance matrix of the simulated data is almost diagonal in Figure 2) in both signal and noise components (of course different structures for signal and noise).
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposes a Bayesian framework for source estimation in M/EEG. The Bayesian framework explicitly estimates spatial and temporal covariance matrices for both signal and noise, before estimating the posterior mean. A majorization-minimization algorithm is proposed to estimate the model, and detailed proofs are provided, including a convergence proof. The model is benchmarked against three source-estimation methods on simulated signals. Finally, the model is used in empirical recordings with a challenging low number of trials.
main_review: **Originality**: The problem of M/EEG source estimation is a well-known and challenging problem. This paper propose a new Bayesian model to solve this problem. The novelty resides in the explicit spatial and temporal priors, where the covariances are estimated jointly with the model. Related work seems to be well cited and discussed.

**Quality**: The model is sound and technically advanced, and the methods used are appropriate. The technical derivations (in supplementary) are thorough and well described. The proofs look reasonable, though I am not familiar with this sort of work, and I have only skimmed through them. The simulation benchmark is short, and only consider a subset of the competing methods, but results are convincing. The empirical result seems impressive. The discussion of the pros and cons of this method is appreciated, although the discussion against type-I methods could have been supported by a benchmark.

**Clarity**: The submission is clearly written and well organized. A large part of the work is located in the long supplementary document, with detailed proofs and discussions. The experimental part is well described, although less well polished than the technical derivations. The figure quality is mediocre. Importantly, the empirical results (Figure 3) are not interpretable without the help of Figure 4-5-6 (all in supplementary). Improving the organization of these figures could help the reader understand the interest of these results.

**Significance**: The model is sounds and seems to be more robust than other models. I am not able to judge if it is a game changer, but it does advance the state of the art in this domain, although the M/EEG practitioners often ignore new advanced models and only use simple ones. To make this work actually used by the community, an effort should be made to release the code with extensive documentation and examples, ideally in Python (the shared code is in a proprietary language (matlab)).

minor comments:
- hyperlinks to references do not work
- L44: M, N and T are not defined
- Fig1: really low resolution
limitations_and_societal_impact: Some discussion is provided about the limitations of the model assumptions.

Reporting computational cost of each method in the experiments would be appreciated.

Societal impact is not discussed, although it is probably not necessary.
review_title: null

Meta-review: The authors present an efficient method to infer models with both spatial and temporal correlations by leveraging Kronoker, sparse, and toeplitz structures. The authors found the method interesting and highly practical as a way to bring additional structure into the regression framework without undue computational burden. While the reviewers did raise a few minor points related to the restrictive model assumptions and the potential for additional comparisons, the authors have provided thorough responses that the reviewers have found adequate. Thus I am happy to recommend this work for acceptance to NeurIPS.
Decision: Accept (Poster)