Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose BlochNet, an approach to reconstructing magnetic resonance fingerprinting (MRF) data. The approach takes the input images, solves for the parameter maps through an inverse problem, and then re-encodes the parameters into multi-contrast images. The encoder is implemented using a neural network and the decoder is implemented using the extended phase graph algorithm. Comparisons are made to other MRF reconstruction techniques across different RF pulse schedule. 

strength_and_weaknesses: Strengths:
- The paper defined the problem well and makes the benefit of QMRI clear.
- The paper proposes a solution to directly incorporating the Bloch equations into the reconstruction

Weaknesses:
- The paper incorrectly claims to be the first to incorporate the Bloch equations into the decoder. This is simply not true. See for example, [1-4]. In addition, the paper is not the first to accelerate EPG to make it usable as a decoder. See for example, [5]. It is unclear how the paper differs from these works, especially with respect to [1] and [5], except that these works can also be used to optimize the pulse sequence. The authors will need to explain how their approach is fundamentally different than these existing works.

[1] Philip K Lee, Lauren E Watkins, Timothy I Anderson, Guido Buonincontri, Brian A Hargreaves, Flexible and efficient optimization of quantitative sequences using automatic differentiation of Bloch simulations, Magnetic resonance in medicine 2019
[2] Beomgu Kang, Byungjai Kim, HyunWook Park, Hye-Young Heo, Learning-based optimization of acquisition schedule for magnetization transfer contrast MR fingerprinting, NMR in Biomedicine 2021
[3] A. Loktyushin, K. Herz, N. Dang, F. Glang, A. Deshmane, S. Weinmüller, A. Doerfler, B. Schölkopf, K. Scheffler, M. Zaiss, MRzero - Automated discovery of MRI sequences using supervised learning, Magnetic resonance in medicine 2021
[4] Evan Scope Crafts, Hengfa Lu, Huihui Ye, Lawrence L. Wald, Bo Zhao, An efficient approach to optimal experimental design for magnetic resonance fingerprinting with B-splines, Magnetic resonance in medicine 2022
[5] Somnath Rakshit, Ke Wang, and Jonathan I Tamir, A GPU-accelerated Extended Phase Graph Algorithm for differentiable optimization and learning, ISMRM 2021

- The paper throws around a lot of jargon surrounding causal and anti-causal mapping, but this makes it harder to read and interpret.
- The paper is heavy in exposition but not heavy in description. This makes it difficult to discern the contribution vs. background material.

clarity,_quality,_novelty_and_reproducibility: The methodology is clear but the novelty is low. Though not provided, I expect the results to be reproducible.

summary_of_the_review: The authors propose an auto encoder for MRF based on NN (encoder) + Bloch equations (decoder). Though powerful, the approach is not new and the paper does not provide any novelty.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a "auto-encoder" for MR Fingerprinting, which is an imaging method in magnetic resonance that captures multiple tissue properties at once (opposed to the "standard" T1 or T2 imaging, which captures a single tissue property, or their weighted variants T1w and T2w which only capture relative weightings of that property). The standard physical model taking these properties to the observed signal are the Bloch Equations, a system of ODEs.

The authors propose to learn an encoder, which (optimally) reconstructs the tissue properties from the timeseries observed measurements, and use the extended phase graph (EPG) method to approximate the forward model (Bloch Eq.s).

They then present results of various baselines (the standard dictionary method, and several deep architectures) on both a phantom and anatomical data (human brain).
strength_and_weaknesses: Strengths:
* This method demonstrates an excellent example of physical models being incorporated into deep architectures, and, conversely, deep architectures informing the estimation of real physical properties.  
* The experimental results are promising, especially in the case of cross-sequence learning.  
* This, in my opinion, is an excellent use case for deep learning in MRI, since the standard fingerprinting already uses a dictionary method (i.e., is already as a field conditioned on some form of learning), and since the inverse problem here is relatively stiff for some parameters/playbacks.  

Weaknesses (extended comments in Clarity section):
* The authors do not describe the EPG method of reconstruction or its use. Even considering appendix A.1/A.2, usage is opaque. Since this portion is describing a critical portion of the method, care should taken so that this entire section is as explicit as possible.  
* The same comment as above, but for the RNN(?) or FC network(?) used as an encoder. This section is completely opaque, except for the fact that some form of learning on the sequence of measurements has occurred. Moreover, section 3.0.2. implies that an RNN is used on the readout timeseries (of fixed length L), while Section 4 refers to BlochNet as (FC-Bloch).  
clarity,_quality,_novelty_and_reproducibility: The manuscript does not describe the procedure of either the EPG-based decoder, or the encoder. Since this is primarily a Machine Learning venue, explaining how the EPG method can be used to form a differentiable forward model in a reasonable amount of time with reasonable accuracy is, in my opinion, the strongest point of the paper. While the math can be relegated to the appendix somewhat, what it's skipping/making assumptions about and how the outcome in A.2 is exactly what we wanted to find would be very helpful to the reader (is it what we wanted to find? seems to be a k-space representation only in space, so we need this also across time?). A.5 includes computational details, but we need methodological details.

A consistent description of the encoder is necessary. Does this encoder take in more than a single voxel? Is it performed in k-space? (Coil correction?) Is it conditioned on the waveforms? How does it generalize well without knowing about the sequence?

Details of the phantom would be a nice inclusion also in the appendix, for those wanting to reproduce the lab results.

Additional comments:
* Should MSE be used without reweighting T1 and T2? These aren't on the same scale as far as I am aware.  
* The description of the method as an "auto-encoder" is somewhat misleading for deep learning readers; this is a supervised method, while most auto-encoders only have priors/regularization and reconstruction loss. I think this is minor, but it might clarify questions here by renaming the paper somehow.  
summary_of_the_review: I think this work is significant and novel, and important for the deep-learning/MRI sub-field, but I think it requires a lot more description of the exact construction of the deep learning method to be publication quality and/or reproducible. It is a good idea, and has the pieces of a good manuscript, but missing details in section 3 and inconsistency with section 4 are strong negatives.

Given a revision fixing these problems (or an explanation of what I have missed/misunderstood), I am willing to change my recommendation.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper introduces a physics-based approach, named BlochNet, for magnetic resonance fingerprinting. Unlike existent works BlochNet directly incorporates Bloch equations in the decoder for improved generalizability. The experiments show improvement compared to conventional and learning-based approaches.
strength_and_weaknesses: Strengths:
-  Including physics of the problem is  important for the medical imaging applications.
- The paper experimental results generally align with earlier reports on the importance of physics-based models (Monga et al, SPM 2021)

Weaknesses:

1) Physics-based models are a well-studied field of work across different domains. The physics-based models are known to be more robust and generalizable compared to data-driven/learning-based approaches (Monga et al, SPM 2021). In fact, for MRI reconstruction tasks such as fastMRI challenge (Muckley et al, TMI 2021), physics-based models have been consistently preferred over data-driven/learning-based approaches. Thus, including the physics of the problem in the network architecture is not a novel contribution, but a straightforward extension of MRF.
 
2) The generalization performance is questionable. While physics-based model improves the performance over learning-based models, they do not necessarily address the out-of-distribution challenges

    a) Based on the findings in Table 1, auto-encoder models such as RNN-RNN do not suffer from the out-of-distribution challenge.  This somehow contradicts the motivation of the study which introduces physics-based modeling to improve the generalization performance. More concrete experiments must be done to show a clear advantage when facing different data distributions. 

   b) Figure 4 results show that BlochNet improves compared to other methods in presence of different RF-Pulse sequences, however, compared to reference it is still far off. Thus, it does not really solve the OOD challenge.

3) Including the physics of the problem tends to have a counter impact on the inference time. A comparative study comparing the inference time is important as inference time plays an important role in medical imaging applications.

4) Please revise the writing as there are repetitive statements (see the last paragraph of the section 2.2)  

clarity,_quality,_novelty_and_reproducibility: The  novelty of this paper is quite limited. The clarity is moderate.
summary_of_the_review: Overall, the novelty is quite limited. motivation and experimental result does not align well. The novelty and experiments need to be improved for ICLR.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper presents a deep learning based reconstruction for Magnetic Resonance Fingerprinting (MRF). The proposed approach based on a AutoEncoder scheme, where the decoder block is inspired by the physics. The so-called BlochNet allows to integrated priori knowledge on the physics driving the generation of MRI, and ensuring that the latent space respects the Bloch equations.
strength_and_weaknesses: Strengths:
1.	The authors contribute to the field of MR fingerprinting by presenting a network that builds on MR physics models and demonstrates a promising generalization performance.
2.	Providing a fast implementation for solving Bloch equations appears to be an impactful contribution to the research community. Sharing the GitHub code improves the reproducibility of their work.
3.	A thorough evaluation of the BlochNet and multiple baseline comparisons was performed on three datasets.
4.	They present appealing visualizations

Weaknesses:
1.	The literature review appears incomplete. An example is the work of Kim et al. who proposed a Bloch equation-based decoder in ‘Fat-saturated image generation from multi-contrast MRIs using generative adversarial networks with Bloch equation-based autoencoder regularization’ published in Medical Image Analysis, 2021. The authors should also check the recent contributions by Jesse Hamilton, especially “Deep learning reconstruction for cardiac magnetic resonance fingerprinting T1 and T2 mapping”, MRM, 2021. This brings questions to the stated novelty of the proposed work.
2.	The discussion of causal and anti-causal mapping appears contextless
3.	There are major grammar and format errors that need to be addressed (e.g. wordy sentences, misplaced citations, inconsistencies in the use of capital and small initial letters as well as blank spaces)

clarity,_quality,_novelty_and_reproducibility: As stated before, there are some missing references. The authors should discuss how the proposed approach is different from state-of-the-art.
The authors aim to provide source code for reproducibility purpose. 

Some minor issues:
•	When claiming superior runtimes, computation times should be reported. 
•	In Figure 3, the subfigures appear slightly shifted, axis labels and color legends are missing, and some text could be sharper.
•	Are the highlighted scores of Table 1 significantly better than the baseline autoencoder approaches? The reported errors seem very close.

summary_of_the_review: The proposed approach seems interesting with the use of prior physics to drive the training of an autoencoder. However, the authors should discuss how their proposed approach is different form missing references, and compare their technique with this state of the art.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes a physics-based appraoch for magnetic resonance fingerprinting data. 

Strength: The paper studies a relevant problem, and the approach of incorporating the physics (the Bloch equations) into the reconstruction algorithm is very sensible and works well.

Weaknesses: The methods is very sensible, but variants of it have been studied an proposed already. Moreover, the reviewers pointed out a variety of issues (such as a lack of comparison to related work, and issues on the clarity/writing). 
justification_for_why_not_higher_score: All four reviewers recommend rejecting the paper for the following reasons: R1 and R3 for it's lack of novelty, R2 and R3 for not being publication-ready, and R4 for a lack of comparison with the state-of-the-art. 
I agree with the reviewers that the paper does not provide a sufficient contribution to the literature to justify acceptance, and is not ready for publication, and therefore recommend rejecting the paper.
Decision: Reject