Paper: Meta-Prior: Meta learning for Adaptive Inverse Problem Solvers (ICLR-2024)

Reviewer comment (Experiments):
Some strong baselines could be considered for comparison.

Author response:
**W2 & Q3. Comparison with additional methods.** We agree with the reviewer that the submitted paper was missing comparisons. We have added a comparison with several methods, namely standard regularization-based approaches, which are widely used in real imaging settings due to their versatility, as well as with the DPIR method, which reaches state-of-the-art performance in synthetic imaging problems. Note that these methods can be seen as unsupervised techniques. up to a single scalar regularization parameter requiring to be fine-tuned. The proposed method performs on par with these different algorithms. Furthermore, we also compare the results obtained by our backbone architecture when trained in a classical supervised fashion on the test task. As expected, this latter approach outperforms the proposed method.

Reviewer comment (Presentation):
In Fig. 2, the similarity between learned and analytic solution seems not close. Absolute difference map may be helpful to visualize the difference.

Author response:
**Q1. Difference between learned and analytic solution.** We appreciate the reviewer's keen observation regarding the dissimilarity between the learned and analytic solutions in Fig. 2. The noted differences are indeed present, and we acknowledge the stochastic nature of the task as a contributing factor. Unfortunately, due to space constraints, we are unable to include an additional figure displaying the absolute difference map. However, we have added a mention to this observation in the text, emphasizing the inherent stochasticity of the task and its impact on the learned solution.

Reviewer comment (Reproducibility):
What are the restrictions on the size of A? Does it have to be the same? How are these achieved when x’s and y’s across different image reconstruction tasks are of different dimensions? More generally, what are the requirements on A across tasks?

Author response:
**1.a Restrictions on the measurement operator A.** Our current implementation has only been trained on image-to-image tasks of the same shape; however, this could be waived in a future implementation. Moreover, there is however no restriction on the dimension and nature of the data used at the fine-tuning stage due to the versatile backbone primal-dual network. For instance, in the MRI experiment, the data of interest lives in $\mathbb{C}$ while the network was trained on variables in $\mathbb{R}$. So there are no direct requirements on $A$ to apply our framework. However, as exposed in Theorem 3.1, the larger the kernel of $A$, the more diverse the task needs to be for the model to learn to complete the missing information and give good performances.

Reviewer comment (Experiments):
It is not clear what are the number of data samples used on each training task; what is the size of the context data (referred to as training data in the paper) for each task versus the query data (referred to as test data in the paper). What about the fine-tuning at test time — what are the number of data used in training/fine-tuning, and how many samples are used for evaluating testing performance, throughout all experiments including tasks seen in training and those unseen.

Author response:
**2.a Clarification on the training data.** We thank the reviewer for pointing this out; we have amended the paper to clarify this matter (see “Training data” as well as experiments’ specific paragraphs). Our network is trained on the BSD500 training and test set combined, containing 400 natural images, and on which all of the 4 inner problems are solved; the outer loss is computed on the test set of the BSD500 images, containing 100 natural images. Finetuning: (i) for SR the model is finetuned on the same dataset used at the inner-level of the MAML task, i.e the 400 natural images from BSD500. The test task is evaluated on the Set3C dataset. (ii) for MRI the model is finetuned on 10 slices from the fastMRI validation dataset, on the same MRI mask that will be used at test time. At test time, the model is evaluated on (different) 10 slices from the fastMRI validation set.

Reviewer comment (Presentation):
The benefits of the presented model needs to be better highlighted.

Author response:
**2.e Benefits of the proposed approach** We believe that the main advantage of our approach lies not so much in the model itself, which is small size and applied to simple imaging tasks, but rather in enabling us to train a model in a situation where no ground truth is available for the specific task considered. A notable instance of this problem is MRI, where the physical acquisition procedure prevents direct access to true data $x$. To learn in such a setting, we leverage tasks for which pairs $(x, y)$ for various sensing operators $A$ are available to propose a principled meta-learning method. Our approach addresses the limitations of handcrafted and implicit priors, and we show theoretically that this framework allows us to learn a prior that is used to complete the missing information in $\operatorname{Ker}(A)$. We have restructured the introduction of the paper to better highlight this motivation.

Reviewer comment (Writing):
The paper structure and writing can be further improved, especially abstract and introduction. It is confusing to understand what is the motivation and what is novelty even after reading through the introduction.

Author response:
**Unclear motivation.** We have amended the abstract and the introduction to clarify the paper’s goal and the main messages. The main motivation of our paper is learning in contexts where no ground truth is available for a given reconstruction task. This context typically arises in MRI, where only partial measurements for a given k-space sampling scheme $A$ are available. When changing the sampling scheme $A$, one does not have access to the true data $x$ but only to the measurement $y$ and the sensing operator $A$. Thus supervised approaches cannot be used to directly train a network in this setting [1]. However, in many cases one has access to pairs $(x, y)$ for different sensing operators $A$. The motivation of our paper is to propose a principled method to leverage these extra data points. Existing methods either rely on handcrafted priors, that have limited performances, or deep implicit prior, that struggle to generalize to non-toy imaging settings. We propose to frame the problem using meta learning. We show theoretically that this framework allows to learn a prior that is used to complete missing information in $\operatorname{Ker}(A)$ and show that this framework allows to give satisfactory reconstruction. Our motivations are thus three-fold: designing a principled framework for explicit prior learning, pulling information from multiple sources (tasks) and accelerating sensing operator adaptation.

Reviewer comment (Theory):
Motivation for learning meta-model. Although it looks interesting from the toy model study, why do we want to learn such a meta-model from different tasks? To converge fast or achieve better results in a new task? But why can the meta prior help to achieve this goal? What is the physical meaning of the learned prior, especially when there are large domain gaps between training and testing tasks such as generalized to MRI imaging? What kind of meta training tasks should be helpful and how to choose such tasks? It is not clear about the grounding support for this proposed method.

Author response:
**Unclear motivation.** We have amended the abstract and the introduction to clarify the paper’s goal and the main messages. The main motivation of our paper is learning in contexts where no ground truth is available for a given reconstruction task. This context typically arises in MRI, where only partial measurements for a given k-space sampling scheme $A$ are available. When changing the sampling scheme $A$, one does not have access to the true data $x$ but only to the measurement $y$ and the sensing operator $A$. Thus supervised approaches cannot be used to directly train a network in this setting [1]. However, in many cases one has access to pairs $(x, y)$ for different sensing operators $A$. The motivation of our paper is to propose a principled method to leverage these extra data points. Existing methods either rely on handcrafted priors, that have limited performances, or deep implicit prior, that struggle to generalize to non-toy imaging settings. We propose to frame the problem using meta learning. We show theoretically that this framework allows to learn a prior that is used to complete missing information in $\operatorname{Ker}(A)$ and show that this framework allows to give satisfactory reconstruction. Our motivations are thus three-fold: designing a principled framework for explicit prior learning, pulling information from multiple sources (tasks) and accelerating sensing operator adaptation.

Reviewer comment (Novelty):
No comparison with baseline methods. The proposed method looks like using a very standard meta-learning framework to train this meta-model. What is the novelty and superiority of the proposed method compared to other meta-learning method such as MAML?

Author response:
**No baseline methods and novelty with regards to MAML.** We have added comparisons with baseline methods for image reconstruction (*see common answer*). The proposed method is an extension of the MAML approach. The novelty in our work is to show how this framework can be adapted in this context, with two main settings (supervised and unsupervised). We further give insights on what this model can learn, and show with a wide variety of experiments that the proposed approach works in practical cases.

Reviewer comment (Experiments):
No comparison with other methods in all the application tasks. As shown in Figure 4 and 6, the results demonstration are mainly the analysis within the proposed method with different finetuning steps or training settings. But there is no comparison with other SR methods or MR reconstruction methods. How can it be argued the proposed method can be a good approach to achieve good results on those tasks? For example, as shown in Figure6, the MRI reconstruction results with acceleration factor = 4 seems to be quite lower-quality than many existing methods either supervised or unsupervised.

Author response:
**Comparison with other methods.** We agree that the initial submission was missing comparisons, which have now been added to the paper. In essence, the proposed method performs on par with the SOTA unsupervised image restoration DPIR algorithm (Zhang et al, 2021) after only 50 steps of finetuning on the test task. However, the proposed model performs less well than its counterpart trained in a supervised setting specifically on the task of interest.

Reviewer comment (Presentation):
I would suggest using a table instead of a figure with so many curves as Figure 5 for results comparison. It is hard to distinguish when many curves are plotted with many overlaps. Showing the network depth may be good additional experiments, but it seems hard to draw some consistent conclusion from these curves.

Author response:
**Clarification of Figure 5.** Following the reviewer’s suggestion, we have replaced Figure 5 with a table (see Table 1) including a comparison with various baselines. We believe that this change improves the clarity of the paper.