reviews: 
Comments: The authors present a method that employs gradient-based meta-learning to adapt models to specific tasks while attempting to preserve few-shot and zero-shot capabilities on other (seen or unseen) tasks. Essentially, the authors penalize the drift (distance) between different sets of base and meta parameters created through a range of interpolations in parameter space.
Strengths: -	The authors present some interesting ideas and insights on the behavior of parameter drift in meta-learning scenarios and how to potentially use distance regularization in this space
-	They also try to improve clarity by providing their algorithms in pseudo code; 

While there definitely are some interesting points to this method, there is significant further work that needs to be done by the authors on their manuscript;   
&#8594; Please see Weaknesses section.
Weaknesses: While I can see that the authors are following a potentially interesting idea and I am aware that some of the ‘issues’ might be due to language difficulties, the current state of the manuscript does in my opinion not warrant publication due to several severe issues; Please see below. 

---
### Clarity of story line, goal and contributions:
The paper is unfortunately rather hard to read and follow; Even reading through the abstract is somewhat confusing and it doesn’t entirely become clear what the authors are going to demonstrate in this work.   
&#8594;  I’d therefore like to encourage the authors to reformulate the abstract and include some more details, underlying intuitions of their approach as well as a glimpse towards the main contributions.  
&#8594; The same applies to the introduction and many further parts of the paper; I am aware that this might be due to language difficulties, but I’d highly recommend the authors to go through their manuscript again and improve the clarity of their argument(s).   
 
Some examples:   
Paragraph 1 start with distribution shifts (time, person (?), environment) – then moves on to few-shot learning with base learners and meta learning; then scaling of models and data, then suddenly to online settings and ‘prior’ continual learning methods; before finally jumping to disentanglement in early paragraph 2.   
This doesn’t create a conclusive story line to follow as there are no real connections between these aspects pointed out, neither does it become clear how these things relate to the work at hand -- therefore the structure would significantly benefit from further (re)work.

---
### Insufficient placement in context of related work:  
The authors simply state a variety of other works without relating them to their own contributions. It is entirely unclear from this section how the authors’ own work is placed in the context of these works, why they are related and to which extend – and how their own work differs. 

---
### Lacking quality & clarity of method description:  
Generally, the authors tend to simply state what is done – but largely fail to provide insight as to WHY certain steps are taken, what the underlying motivation is, what we should expect, what they want to show, etc.;   
Some examples (not comprehensive):  
- The authors state in Sec. 3.1 that the pre-trained model **already performs well across all datasets**; yet, they train on MSCOCO to **overfit** to the point where the model **does no longer perform well** on the other sets; In other words, the authors **create their own problem that they want to then solve**. While this can be done, I am entirely **missing the intuition/justification that is underlying this approach**.
- Why don’t the authors start from the ‘well-performing’ pretrained backbone and try to further improve on the small datasets? Why don’t the authors choose to start from a backbone that’s solely trained on MSCOCO to begin with (which would be more alike other continual learning works, i.e. iteratively learning new skills). 

- While I can see the idea of comparing “functional distances” based on model performance in Table 2, what is the underlying justification that the cosine distance between the flattened parameter sets directly represents the ‘distance’ between these datasets (Table 1)? What happens if you simply choose another random seed for the SAME dataset – wouldn’t you also end up with a different set of parameters that simply converged to a different local minimum, and all distances might therefore change? 
- Section 3.1 would equally benefit from some more details regarding underlying motivation, intuition etc. – as would Section 3.2. 
- Section 3.2 suddenly splits ONE dataset into several tasks (Split-MSCOCO), whereas up to this point, different datasets were considered – again, it would help to provide insight as to WHY this experiment is performed. 
- Section 3.3. describes investigations on the drift of meta parameters that can be tolerated and states at the end that the authors are interested in “maximizing the radius of the parameter subspace […]” – it would however again significantly help the reader WHAT the underlying intuition and purpose of the experiments is.

---
### Some inconsistencies throughout:
E.g.:  
- Section 3.2 states that “we can project the task-specific representations… within the meta parameters to the base parameter space”, but the authors already stated a few sentences earlier that “meta-parameters and base parameters reside in the same parameter space”. This contradicts itself since there is no need to project anything if both already live in the same space.

---
### Unstated implications & Limitations: 
Some examples:   
- Algorithm 3 shows that the base parameters for all tasks have to be stored (line 6.); Further, an additional set of interpolated base parameters is stored for each task throughout the interpolation procedure (line 15). &#8594; Comments on the additional required memory, especially for large models and many tasks feels necessary & helpful!
- Algorithms 3 loops through all tasks to determine base parameters, then through the tasks again to compute task losses, followed by an additional nested loop over all random interpolation params ‘S’ (here 1000) with a set of interpolation constants within; &#8594;  Commenting on the train complexity in terms of runtime would be helpful as well.  
 
&#8594;  Note that discussing this becomes imperative especially because the authors claim in the conclusion that their method “scales to large models”
Recommendation: 1: strong reject
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 1 poor
Clarity: 1 poor