Gradient-Based Transfer Learning
Authors: Gustaf Tegnér
Alfredo Reichlin
Hang Yin
Hedvig Kjellstrom
Mårten Björkman
Danica Kragic
Venue: Submitted to ICLR 2023 (2023)
Keywords: meta-learning
gradient-based meta-learning
transfer learning
representation learning

Abstract: We formulate transfer learning as a meta-learning problem by extending upon the current meta-learning paradigm in that support and query data are drawn from different, but related distributions of tasks. Inspired by the success of Gradient-Based Meta-Learning (GBML), we propose to expand it to the transfer learning setting by constructing a general encoder-decoder architecture that learns a map between functionals of different domains. This is achieved by leveraging on the idea that the task-adapted parameters of a meta-learner can serve as an informative representation of the task itself. We demonstrate the proposed method on regression, prediction of dynamical systems and meta-imitation learning problems.

Under review as a conference paper at ICLR 2023
GRADIENT-BASED TRANSFER LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
We formulate transfer learning as a meta-learning problem by extending upon the current meta-learning paradigm in that support and query data are drawn from different, but related distributions of tasks. Inspired by the success of Gradient- Based Meta-Learning, we propose to expand it to the transfer learning setting by constructing a general encoder-decoder architecture that learns a map between functionals of different domains. This is achieved by leveraging on the idea that the taak-ad parameters of meta-learne can serve as an informative repre- sentation of the task itself. We demonstrate the proposed method on regression, prediction of dynamical systems and meta-imitation learning problems.
INTRODUCTION
The ability to quickly adapt to unseen conditions is a necessary skill for any intelligent system. It provides the means to generalize outside of the training conditions well as the capacity to extract unobservable features affecting the learner (Lake et al. (2017)). Adaptation to a new task involves two steps. The first is inferring the characterizing information of the task at hand. The second is regressing the function representing the task. The importance of this ability is reflected in the considerable volume of work conducted on the matter in the past years e.g. Hospedales et al. (2021): BBn--avvis et al. (2006); Ljung] (2010). The field of meta-learning provides the means to unify these two steps and learn them simultaneously and fully data-driven (Huisman et al. (202T), ) The learning process comprises multiple datasets representing different conditions, or tasks, the learner is concurrently exposed to. Adaptation is performed by extracting the relevant information about each task from small set of data sampled from the task.
In this paper we consider the case of transferring knowledge using a small set of data from task to another, different, task. In this regard, we build upon the framework of few-shot learning (Wang) ict al. (2020)) This can be summarized as estimating an optimal learner for any task with the fewest data samples possible. Recent work has explored the case where the data used for the adaptation and the downstream-taste data are subject to a distributional shift in their domain, referred to as support-query-shit (Bennequin et al. (202T)) Here, meta-transfer where the shift can take place on both the domain and co-domain of the underlying function generating the data. This brings beyond the problem of domain-shift and into the more general notion of learning to transfer between support task and query task.
assume the
general formulation of
The need for transfer emerges multitude of situations. Sequential decision-maie making problems are one of them. Real-world dynamical systems, for example, are often only partially observable. They require an initial exploration phase gather the necessary information before estimating a suitable policy. In this case, we would need a way to transfer the knowledge acquired from the dynamics of the system to the estimation of the target policy. That is, transfer between dynamics prediction model to the estimation of policy in control problem. Moreover, transfer learning can be used in situations where we have access to labeled data of simple problem but would like to solve a more complex, but related, problem. For example, transfer from single inverted pendulum to double pendulum with the same dynamics e.g. same length of the poles, same gravity and friction coefficients.
To this end, we present an approach to transfer learning through adaptation. Inspired by Gradient- Based Meta-Learning (GBML) we propose method for meta-transfer learning in a general encoder- decoder model. This can be used independently of the shift between the support task and the query task and is agnostic architectural changes between the meta-learner and the base-learner (see Fig-

---

Under review as a conference paper at ICLR 2023
C(0,D3)
Li DQ)
V.2
V
Visual depiction of the proposed model. The representation of the task is the gradient, VaL, of meta-learner model i.e. green arrow on the blue loss surface the left. This representa- tion is then mapped through Ma (light blue arrow) to the parameters, v of the main network which optimal for the given task (minimizes the orange loss function the right)
Figure I:
ure [. The main idea this work that the parameters of a learner that is optimal for a given task contain all the relevant information of such task (Tegérr et al. (2022). The proposed model learns a map from the gradients used to adapt the parameters a meta-learner to the parameters of a base learner. This, in fact, is map between functions that has proven to be effective in different contexts e.g. Xu et al. (2020); Dupont et al. (2022). We argue that representing the task's parameters as the gradients of the meta-learner is robust to noise and bias in the data. We empirically support this claim with number of experiments on synthetic regression, dynamical system prediction and meta-imitation learning.
Our contributions are as follows:
We extend the formulation of support-query shift the problem of transfer learning.
We describe meta-transfer learning method that builds upon previous gradient-based methodologies.
We provide an empirical evaluation of the advantages of gradient-bascd task representations variety of problems.
RELATED WORK
In this section we review the relevant literature regarding adaptation methods. The idea of adapting the learning system has been widely studied the past years (Naik Mammone (1992): y Bengio al. (1990); Hochreiter et al. (2001). Adaptation is performed using data-points that uniquely characterize the task. Different approaches can be included in this definition depending on the framework they abide and the assumptions they make about the adaptation process.
Transfer Learning. It refers to the problem of learning algorithms to extract knowledge from one task solve second task (Weiss et al. (2016); Zhuang al. (2020); Pan & Yang (2009)). These methods cannot be considered to perform general adaptation strategies. However, they require the identification of useful information for task from different distribution. They are generally limited to two tasks only and most involve aligning the distributions of these two tasks. In Wu & He (2022) they propose the use of heta-lea for transfer learning problem. The method limited to matching the empirical distribution of dynamic source and target tasks.
Parameter Identification. more general form of adaptation to dynamical systems can identi- fied in the early work on system identification (Aström & Eykhoff (171)). More precisely, parame- ter identification refer to the estimation of unobservable parameters influencing the dynamic system considered from a sequence of observations. Most of these studies however consider only one of the two required steps for adaptation. In fact, they assume to know the law governing the process and estimate the conditional parameters (Bhat al. (2002); Yu et al. (2017)), impose suitable inductive bias to guide the learning process (Sanchez- (Sanchez-Gonzalez et al. (2018) (1) or use a hybrid approach to learn residual of imperfect but known system (Ajay al. 2019)).
2

---

Under review as a conference paper at ICLR 2023
Optimization-Baser Meta-Learning. Meta-learning is concerned with estimating both of the quired adaptation steps from data alone. Optimization-aasee methods do this by performing an adaptation step the learner itself (Ravi & Larochelle| (2016)). Of these, sradient-Basee Meta- Learning is a particular case where the adaptation is performed by a gradient descend step on the parameters of the learner (Finn et al. (201))) This family of methods have heen shown be univer- sal function approximators (Finn & Levine (2017)). One shortcoming of these methods is that they can only be used on problems where the architecture of the function used infer the gradient the same as the adapted learner. Indeed, the only works considering form of transfer assume a shift in the imput space (Bennequin et al.] (2021); Du et al. (2020): Jiang al. (2022). In contrast to past work, our method can be applied in the case of the adaptation data being of different nature with respect to the final task.
Model-Based Meta-Learningg A second approach to meta-learning is to learn a model to output directly the adapted learner for the task. For problems requiring a specific adaptation strategy, in fact, the two steps be coupled together and learned with unique model (Xu et al. (2019); Li et al. (2018). More general is, instead, the use of a parallel neural network, conditioned on the adaptation data, output directly the parameters of the adapted learner. HyperNetworks one of the most commonly used methods for this (Ha et al. (2016). This has been done using amortized inference (Gordon al. (2018)), gradient-based (Rusu et al. (2018); MMnnkhdalai Yu (2017) or outputting a conditional to the learner (Kirchmeyer et al. (2022). Similarly to our work, (iaa al. (202I)) have used HyperNetworks to estimate the unobservable properties of dynamical systems, In contrast, they require the use of a feature extractor when dealing with high- high-dimensional inputs. Overall, these black-box approaches can be particularly expressive but suffer generalization performances and are subject to statistical noise. Another line of work is the Neural Process Family Garnelo et al. (2018). Kim et al. (2019)). contrast to HyperNetworks, they generate functions by conditioning and also incorporate uncertainty through a variational approach. [Wang et al. (2021) meta-learning framework where they learn to transfer between different tasks from the NLP domain. In contrast to work, they use the Fisher Information Matrix of their base learner to define the task representation.
considers model-basee
PRELIMINARIES
We situate transfer learning as a Sarming-to-leans problem. As necessary preamble, we review the general formulation of meta- veta-learning and describe two specific instances of as GBML and HyperNetworks respectively.
3.1 META-LEARNING
Meta-Learning describes a family of algorithms that are designed for learning-to-ler to-lerr.. Given a space of tasks, meta-learner utilizes previous knowledge efficiently learn new tasks using only limited number of data samples. A task can equivalently be seen as a dataset or a function. Formally, we define a task Ty C X x Y as collection of input-output pairs defined by an underlying, unknown function f X + Y. In other words, T, {xx f (x)) j E X]. From this definition, we can denote the space of tasks over a function space as Tr. In fact, the function f uniquely identifies the task Ty, Throughout the paper we drop the subscript f whenever the tasks functional dependence on f is not of importance.
In the standard supervised learning setting, we aim to learn a function So with parameters v V C Ted that approximates a function f through supervised loss L(T,, 1)) The purpose of meta- learning is, instead, to find set of optimal parameters from only a small dataset Dy Ty- From the notation defined above, the meta-learning methodology can be formalized as learning parameterized update function Ma : T x θ - V that maps single task Ty and some prior 0 E θ to the updated optimal parameters va of fw. The optimization problem then be stated as:
min ET, 0.9
[[D) })
s.t. b= McM
(1)
Here, D3, DJ refer to support and query set that sampled without replacement from task Ty-
3

---

Under review as a conference paper at ICLR 2023
3.2 GRADIENT- BASED META- LEARNING
The framework described above is general and fits a variety of meta-leaming methodologies. These methodologies mainly differ in how they implement the update function M. In particular, GBML uses as a prior another set of parameters i.e. E θ C R2. This common set of parameters used as the initialization among tasks such that any task can be learnt by only few gradient-steps on limited number of samples. In the general case, the update function can be expressed as:
MD
MVoL(D),
Here, M Rddd is a learnable preconditioning matrix which facilitates the gradient descent. In particular, we have e = M in this case. For example, a diagonal preconditioning corresponds to learnable learning rates used in Meta-SGD (Li et al. (2017)) while full-rank matrix corresponds to Meta-Curvature (Park & Oliva (2019). Other forms of preconditioning have been studied in (Lee & Choi (0018); Flennerhag al. (2019)).
3.3
HYPERNETWORKS
GBML incorporates the inductive bias in that adaptation to a new task necessarily implies an opti- mization procedure. From the general formulation expressed in Equation] [ this does not necessarily have to be the case. An alternative approach is to consider the map Ma D x è -> v directly as a parameterized neural network. As such, the network takes input the task and possibly some ad- ditional parameters and outputs the task-adapted parameters yb directly. To this end, .M constitutes HyperNetwork (Ha al. (2016) which are networks whose output are the weights of another neural network denoted as the main network. This formulation is also consistent with recurrent- based leaaners that implement a learning algorithm through recurrent neural network (RNN)
(Santoro et al. ()
The update function M, in this case, can be formulated with a general auto-encoder structure i.e. Ma = Mb, cM HyperNetworks take as input the support dataset Dÿ ~ TT of task by encod- ing the N input-output pairs (zf yr ) using non-linear function hのg To ensure permutation- invariance, an appropriate aggregation function Σ the task can then be formulated as:
can be used. The latent representation z E g of
2 = M (Df) = Eitho (1 ue ())
(3)
This representation is then passed through the decoder MD to output the parameters va of the main network 9a.
4 META TRANSFER
Transfer-Learning is concerned with learning a specific target task, given knowledge of a source task. To achieve this, one can utilize some inductive bias c.g. the assumption that representations leart in the source task can in turn be useful to learn the target task. We now ask the question this inductive bias for transfer can be learnt as well. Instead of single source and target, we consider joint distribution over pairs of tasks and aim to learn to meta-transfer. In the meta- learning formalism we have presented up until now, we have made the assumption that the support and query set are samples from the same task w.r. a functional f i.c. D Dy Ty. To address the problem of meta-transfer, we extend the meta-learning formulation by considering support and query to be defined over different function spaces In this respect, we have D T, and D& T, with if f To learn an efficient adaptation on the query set, there needs to be an explicit relationship between support and query. We define this as an unknown map T from function to function such that y = T()). In particular, when T = / we fall back to the case of standard meta-learning. On the other hand, the meta-learner needs to learn this functional dependency as well if A I. In the next section we describe the proposed method to achieve this.
4.1 METHOD
When the two functions ff and g require a different architecture for their parametric approximation, standard GBML methods cannot be used. In this section descrihe extension of GBML to
4

---

Under review as a conference paper at ICLR 2023
handle such cases. In particular, propose to approximate T by learning map from function space to function space. Let fe be a parameterized neural network and Dÿ = -xx, y ) be the support data sampled Ty. We argue that a good representation of the function f is a function itself. In similar notation as in Section 3.3 we want to construct an encoder M of the support set. We attain this by defining M as the gradients of fa w.r.t. to a chosen loss function L on the support. However, representing a function through its parameters bears with a problem of dimensionality.. Since the parameters 0' F are of neural network, they will possibly belong to a very high-dimensional space. To solve this, we adhere to only optimizing over a subspace of the full parameter space.
In practice, there are two ways of achieving this. The first is to modulate the gradients through conditioning variable that is concatenated with the input. The second approach is to use hypernetwork he, Z - θ from low-dimensional latent-space Z C && to the parameter space of the learner More specifically, the first approach was explored in the meta-learning literature as CAVIA (Zintgraf et al. (2019)) while the second approach corresponds to the method employed in LEO (Rusu et al. (2018)). This last method can be further divided into two variations based on the implementation of the function hag,. This can be, in fact, either a linear or non-linear neural network. The linear approach would effectively imply a linear projection of the gradients to a low- dimensional subspace.
R gives task representation in a similar vein as equation This representation of the function I can thus be extracted by means of M using one of these three methods summarized below:
. Context: We concatenate the input to learner f with a parameter 2 E R to modulate the output. Thus:
M (D) ER,yDD, [ [.f(fo(z, z), [z, 0]
Non-Linear:
The task adapted parameters are modulated through a latent parameter & This is achieved through the use of a hypernetwork ha, mapping from the latent space
to parameter space:
M (D), Ee, (s)
{ [=
(5)
Linear: In the case of linearity we rewrite the hypernetwork as V = h The matrix
V E p is employed to linearly project the gradients to lower-dimensional subspace. We thus calculate the l directional derivatives w.r.t (vi,...,, Vk V:
M
Ex,yDD, (o = [V,8]
The representation of the support task can then be used to estimate an optimal learner on the related downstream task Tg. This, in turn, requires decoding this representation to a set of parameters that are optimal on the query dataset DΩ For this, we employ a decoder network MM on that outputs the parameters v of neural network Jo. In summary, our update function defined as
M(D,) MM
We train the model end-to-end by optimizing the loss of Iø on the query set D2 on every training task through mraiient-dsceet The final objective of the model can thus be formulated as:
min TT (C(D,, b)] s.t. + =
(8)
5 EXPERIMENTS
We validate our approach on three different classes of problems involving regression, dynamics prediction and imitation learning. For each these problems, we evaluate the ability to transfer in the presence of increasing amount of noise in the support. We further assess the representational capability of our model by learning to predict the ground-truth task parameters from the learnt representation. Finally, we evaluate model's 's capacity to learn high-dimensional representations by considering a maze environment which the ground-truth task parameters are potentially high- dimensional and unknown.
5

---

Under review as a conference paper at ICLR 2023
5.1 BASELINES
We consider different encoders ME as our baselines. In past work (Xian et al. ). Garnelo et al. 20188). the mean pooling function is used. For comparison, also consider the MAX operation. For context, mean pooling would correspond to DeepSets, (Zaheer et al. (2017)) while max pooling would correspond to PointNet (Qi al. (2017). Furthermore, we consider an encoder based on the transformer architecture (Vaswani et al. (2017)) in which we find weighted average of all the samples in the support where the weights are the pairwise dot-produets between encodings of the data points. We refer to the different aggregation schemes pooling methods in our experiments. Lastly, we consider deep-kernel architecture based on MetaFun (Xu et al. (2020)) Here, points of the query are directly compared to points from the support based on learnt kernel-function. As such, we do not achieve an intermediate latent representation of our task but rather we directly out- put a function f defined as f f= Ei k(,,,, )r(a;; u.") with k, as parameterized neural networks. For the gradient-based encoders, we consider the three different methods outlined in Equations 4 To enable a fair comparison between the methods, we employ the same decoder network for all models except MetaFun. Implementation details can be found in the Appendix. We use the same subspace dimension &o for all models and let & equal the true dimension of the task-space when known as we found this is sufficient to achieve a good performance.
5.2 SYNTHETIC REGRESSION TASK
As proof-of-concept, we consider the problem of transfer learning between two sinusoidal func- tions. We consider the support data to be regression tasks drawn from y = A cos(x + b) with A E (0.1,5.0) and [-T, The corresponding query tasks are constructed from sine wave with the same amplitude A and phase as the support task. The task is then to essentially extract the task-pare misk-parameters A, b from the support learn the query task. In this experiment, we highlight our methods robustness to white noise in the support. The results can be seen in Table [ As the amount of noise increases, the performance of gradient- based encoders remains constant while other pooling methods eventually become unstable. Figure 2 shows qualitative results. The figure shows the sinusoids found through the linear- projectoooottionn of the gradient compared the ground-truth and MEAN and MAX pooling. We evaluate the models on 100 different samples of noise and plot the mean and standard deviation. The results show the robustness of our method even up to noise with standard deviation of 4.0.
Baio:
Soad Tun
Figure 2: Sinusoid regression for different levels of noise. Using the gradients as an encoder stays invariant to the amount of noise in the support data.
5.3 DOUBLE INVERTED PENDULUM
For our next experiment we consider the more complex, real-world scenario learning the dynamics of physical system. The system consider is the double inverted pendulum. We hypothesize that from observing the dynamics of single pendulum, one can infer the dynamics of a double pendulum that shares the same global physical parameters and object properties the single pendulum. The single and double pendulum are simulated using the Mujoco environment (Brockman et al. (2016)) and involve pendulum attached to cartpole which can move left and right. Our support task thus consists of state-action-star triples drawn from the single pendulum. Concretely, we have g$ = S x A and ys S with S defined the state of the pendulum denoted by its position, velocity, angle and angular velocity, (s, f. 0, 0) and A [-3,3] representing the force applied to

---

Under review as a conference paper at ICLR 2023
Model
0.0
0 0.5
1.0
0 1.5
POOLING GRADIENT
-CNTTXT
0.23+0.01
0.35
0.49
0.61
-NON-LINEAR
0.41
0.46+0.05
0.57+0.09
0.69
-LINEAR
4.005
0.36
0.43+0.08
0.51
MEAN
0.24
0.25
0.44
0.86
.MXX
0.13+0.06
0.22
0.61
1.31
-TRANSOORMRR
0.26+0.01
0.80
1.34
2.30
DEEP KERNEL
0.75
0.87 10.04
1.1240.03
1.58
Table I: Results for the cosine-sine experiment. The top three rows compares different methods of encoding the gradients based on LINEAR, NON-LINEAR and CONTEXT methods. We compare all models against different standard deviations of noise in the support (o)). From the results, gradient- based encodings are more robust as the amount noise increases.
the base cart. The corresponding query task is forward dynamics prediction the double pendulum. Hence, X9 and Yy are state-action triples as before. We experiment with varying degrees of noise confirm our models robustness properties. For the support data, we add noise ÿ y² + e with e ~ NN,00 r²). ). We experiment with various E [0.4, 1.0, 2.0, 3.0) respectively. The results for the forward dynamics prediction are shown the Appendix in Table To further confirm our findings on noise robustness, we perform an additional experiment where we instead consider the task of regression to physical parameters of the single pendulum. We plot the MSE to the physical parameters (pendulum length and gravity) against the standard deviation of noise in the support. We evaluate each model 100 times and plot the mean and standard deviation. The results are shown in Figure 3 We can note similar performance across all models for low levels of noise. As the amount of noise increases however, the MAX aggregation quickly explodes. The MEAN aggregator is more robust but similarly is also not unaffected by noise in the support. In this experiment, we also note that methods based on non-linear projections of gradients such as CONTEXT and NON-LINBAR show better performance than the linear projection.
5.4
IMITATION LEARNING
Model
o 0.0
0 1.0
0 2.0
= 4.0
GRADIENT
CONTEXX
4.075
4.22010.154
4.321 19.2220.22
4.361 10.152
NON-LINEAR
4.255.0.010
4.247=0.144
4.295
4.096
LINEAR
3.999
4.0624+0.00
4.173=0.106
4.052+20.19
POOLING
.EAN
3.929 10.061
3.930.1..04
3.955.1..34
3.370
.MAX
4.
11.080.155
4.027 10.157
3.95840.322
3.13911.060
TRANSFORMER
00122
4.19944:222
1.00740.104
-30.5644.2.400
DEEP KERNEL
0.08610.086
-0.001
0.099
-0.19221..14
Table Final reward for the imitation learning experiment. Higher is better.
We conduct experiment probe models ability infer optimal policy for given MDP given only few observations from the environment. We consider a modification of the Mujoco Ant environment where vary the length of the legs as testing ground (see Figure|4and Appendix).

---

Under review as a conference paper at ICLR 2023
Up C2
ag G7AB454
GANAMAA
66 OA
Figure 4: Visualization of the modified Ant environment. We vary the length of the upper and bottom parts of the leg to create different conditions for the control task.
The goal of this experiment is test the models ability to transfer between dynamics data and pol- icy estimation. Here we are not interested in in- ferring an optimal exploration strategy but rather assume that the dynamics data contains all the relevant information to infer the optimal behavior the agent. We avoid the use of reinforce- ment learning loss as it has been shown that com- tools can't compute the second derivative of Monte-Carke expectations (Rothfuss et al. (018): Focrster et al. (2088). The target loss is thus behavioral cloning of second policy trained with privileged information. We train PPO (Schulman et al. (2017)), conditioned on the ground-truth physical parameters. Our query task is then from a given state to infer the action given by the optimal policy. We condition our meta- imitation learner by considering a trajectory (st, de; st+1)==1 gathered from random policy as our support data. We train for 100 epochs on dataset of 100 different tasks with 7 = 30 number of points in the support. Each task constructed by sampling an upper-leg length and ankle length from (0.2, 0.6). We experiment with inputting varying amount of noise in the support data during test- ing to validate our models robustness to noise. We show the final reward after one episode for the methods in Table We note that even with noise infused on the support consistent high reward while for our baselines they slowly decline. With random noise added with 8.0. the transformer-based architecture achieves large negative reward while the deep-kernel method failed to transfer in all cases.
Mean Contoc
Liverr
Nor Lincar
2V23
D61
mon iikomatic-differentiane
Figure MSE of regression to the ground- truth task parameters from finite set of in- teractions with a single pendulum.
method achieves
5.5 MAZE EXPERIMENT
We implement simple maze environment defined over a 2x x N grid. We define the starting point of our agent in upper left corner and the goal to reach the lower right corner as efficiently as possible. To generate different mazes, randomly place obstacles on the grid in such way that the maze is always solvable (see Appendix for further details). The optimal policy is in this case deterministic and found through depth-first search. For the support data let X C Z? be positions of the grid and S = {0, 1} be binary variable indicating if position in the grid is occupied or not. We experiment with varying the width N of the maze by letting N [6, 12, 24].
The results can be seen in Table 3 In contrast to previous experiments, the advantage of deep kernels become apparent in this task. Furthermore, the linear method achieves significant performance gain over the other methods.
DISCUSSION
6.1 THE ADVANTAGE OF GRADIENT-BASED ENCODERS
The empirical gradient of function VL is unbiased estimator of the true gradient. As such, it possesses form of consistency as it converges in distribution to the true expected gradient of the loss over the entire data-space. This, results in statistical advantages over black-box representations.
8

---

Under review as a conference paper at ICLR 2023
Model
MAZE-SIZE
MAZF-SIZE
MAZE-SIZE. 24
POOLING GRADIENT
CONTEXT
00.337+0.022
-0.368
-0.435
-NON-LINEAR
0.000
-0.348
-0.415+0.015
-LINEAR
0.000
-0.03640.020
-0.355+0.041
.EAN
0.000
-0.291
-0.39640.003
.MAX
0.000
-0.156=0.090
-0.386+0.017
"TRANSFORMER
0.028
-0.275+0.029
-0.40040.025
DEEP KERNEL
0.00011.000
-0.257 :000010
-0.42810.010
Table 3: Final reward as measured by the normalized distance to goal position for the maze exper- iment. The results show that a linear projection can efficiently encode and transfer the information gathered from the exploration phase while other gradient-basee methods fail compared to the base- lines.
One being resilience in overfitting and better generalization to out-of-domain tasks, as shown in Finn & Levine (2017). Another being robustness to white noise in both the input and the output of the support data, as empirically shown in this paper. This robustness to noise makes the use of gradient as a representation appealing for real-world applications where noisy observations and faulty labeling often unavoidable. Moreover, the gradient carries a semantic meaning that will always point in the direction of steepest descent given the data.
6.2 LIMITATIONS
Utilizing gradient-information in an end-to-end manner requires computing the second order deriva- tive during training. This can be prohibitively expensive in large-scale experiments. Using first-order methods such as Reptile (Nichol et al. (2018) could potentially be incorporated into our method to alleviate this concern. second limitation the dimensionality of the parameter space. When deal- ing with large networks this might cause memory-related issues. Lastly, even though theoretically sound, second-order Monte Carlo expectations cannot currently be handled by automatic differ- entiation tools. This prevents the use of any GBML method with loss functions involving these expectations e.g. reinforcement learning losses.
7 CONCLUSION
In this paper we have proposed general framework for transferring knowledge from one task to another from adaptation perspective. To this end, we have described family of methods based on the intersection between GBML and model-based techniques. Furthermore, we have explored the use of gradients sentations. We have empirically demonstrated the advantages of this representation on number of experiments. Such advantages are especially noteworthy in case of statistical errors in the adaptation data like the presence of white noise. These results not only reinforce the advantages gradient- based meta-learning but exemplify how the same methodology can be extended novel problems where gradient-based methods have previously been unexplored.
task representation and the advantages of such with respect to other repre-
As future line of work, one can further connect method the neural process family by ex- plicitly incorporating uncertainty measures into the encoder. This can for example be achieved by imbuing MAML in probabilistic framework done in Finn al. (2018) Another possible exten- sion is the use of hierarchical GBML methods to handle complex distributions of tasks.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Anurag Ajay, Maria Bauza, Jiajun Wu, Nima Fazeli, Joshua B Tenenbaum, Alberto Rodriguez, and Leslie Kaelbling. Combining physical simulators and object-basede networks for control. In 2019 International Conference Robotics and Automation (ICRA), pp. 3217-3223. IEEE, 2019.
Karl Johan Aström and Peter Eykhoff. System identification-aa survey. Automatica, 722)1223-122 1971.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
Yoshua Bengio, Samy Bengio, and Jocelyn Cloutier. Learning a synaptic learning rule, Citeseer, 1990.
Etienne Bennequin, Victor Bouvier, Myriam Tami, Antoine Toubhans, and Céline Hudelot. Bridging few-shot learning and adaptation: new challenges support-query shift. In Joint European Con- ference Machine Learning and Knowledge Discovery in Dutabases, pp. 554-569, Springer, 2021.
Kiran S Bhat, Steven M Seitz, Jovan Popovié, and Pradeep K. Khosla. Computing the physical parameters of rigid-body motion from video. In European Conference on Computer Vision, pp- 551-565. Springer, 2002.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojeiech Zaremba. Openai gym, 2016.
Yingjun Du, Xiantong Zhen, Ling Shao, and Cees GM Snoek. Metanorm: Learning to normal- ize few-shot batches across domains. International Conference on Learning Representations, 2020.
Emilien Dupont, Hyunjik Kim, SM Eslami, Danilo Rezende, and Dan Rosenbaum. From data to functa: Your data point is function and you should treat it like one. arXiv preprint arXiv:2201.12204, 2022.
Chelsea Finn and Sergey Levine. Meta-learning and universality: Deep representations and gradient descent can approximate any learning algorithm. arXiv preprint arXiv:1710.11622 2017.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp- 1126-11333 PMLR, 2017.
Chelsea Finn, Kelvin Xu, and Sergey Levine. Probabilistic -odel-agnostic meta-learning. Advances in neural information processing systems, 31, 2018.
Sebastian Flennerhag, Andrei A Rusu, Razvan Pascanu, Francesco Visin, Hujun Yin, and Raia Hadsell. Meta-leariin with warped gradient descent. arXiv preprint 1rXhv:1909.00025 2019.
Jakob Foerster, Gregory Farquhar, Maruan Al-Shedivat, Tim Rocktäschel, Eric Xing, and Shimon Whiteson. Dice: The infinitely differentiable monte carlo estimator. In International Conference on Machine Learning, pp. 1529-1538. PMLR, 2018.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and SM Ali Eslami. Conditional neural processes. In International Conference on Machine Learning, pp. 1714-1713. PMLR, 2018
Jonathan Gordon, John Bronskill, Matthias Bauer, Sebastian Nowozin, and Richard E Turner. Meta- learning probabilistic inference for prediction. arXiv preprint arXiv:1805s 1805.09921, 2018
David Ha, Andrew Dai, and Quoc V Le. Hypemetworkss. arXiv preprint wwXXiv160900910000 2016.
Sepp Hochreiter, A Steven Younger, and Peter Conwell. Learning to learn using gradient descent. In International conference artificial neural networks, pp. 87-94. Springer, 2001.
10

---

Under review as a conference paper at ICLR 2023
Timothy M Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 2021.
Mike Huisman, Jan N Van Rijn, and Aske Plaat. survey of deep meta-learning Artificial Intelli- gence Review, 54(6):4483 4541, 2021.
Siyang Jiang, Wei Ding, Hsi-Wen Chen, and Ming-Syan Chen. Pgada: Perturbation-guiied adver- sarial alignment for few-shot learning under the support-query shift. In Pacific-Asia Conference on Knowledge Discovery and Data Mining, pp. 3-15. Springer, 2022.
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive neural processes. arXiv preprint 1rXiv:1901.05761, 2019.
Matthieu Kirchmeyer, Yuan Yin, Jérémie Donå, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing to new physical systems via context-informed dynamics model. arXiv preprint arXiv:2202.00 00.00.008899 889, 2022.
Brenden M Lake, Tomer D Ullman, Joshua Tenenbaum, and Samuel Gershman. Building machines that learn and think like people. Behavioral and brain sciences, 40, 2017.
Yoonho Lee and Seungjin Choi. Gradient-based meta-leaming with learned layerwise metric and subspace. International Conference on Machine Learning, pp. 2927-2936. PMLR, 2018
Jue Kun Li, Wee Sun Lee, and David Hsu. Push-net: Deep planar pushing for objects with unknown physical properties. In Robotics: Science and Systems, volume 14, pp. 1-9, 2018.
Zhenguo Li, Fengwei Zhou, Fei Chen, and Hang Li. Meta-sgd: Learning to learn quickly for few- shot learning. arXiv preprint arXiv:1107.09885, 2017.
Lennart Ljung. Perspectives on system identification. Annual Reviews in Control, 34(1):1-12, 2010.
Tsendsuren Munkhdalai and Hong Yu. Meta networks. In International Conference on Machine Learning, pp. 2554-2563. PMLR, 2017.
Devang K Naik and Richard Mammone. Meta-neural networks that learn by learning. In /Proceed- ings 1992] IJCNN International Joint Conference on Neural Networks, volume 1, pp. 437-442 IEEE, 1992.
Alex Nichol, Joshua Achiam, and John Schulman. first-order meta-leariing algorithms. arXiv preprint arXiv: 1803.0099, 2018.
Sinno Jialin Pan and Qiang Yang. A survey transfer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345-1355 2009.
Eunbyung Park and Junier B Oliva. Meta-curvature. Advances in Neural Information Processing Systems, 32, 2019.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 652-660, 2017.
Sachin Ravi and Hugo Larochelle. Optimization as a model for few-shot learning. 2016.
Jonas Rothfuss, Dennis Lee, Ignasi Clavera, Tamim Asfour, and Pieter Abbeel. Promp: Proximal meta-policy search. arXiv preprint / 1810.06784, 2018.
Andrei A Rusu, Dushyant Rao, Jakub Sygnowski, Oriol Vinyals, Razvan Pascanu, Simon Osin- dero, and Raia Hadsell. Meta-learning with latent embedding optimization. arXiv preprint arXivv1807.0596). 2018.
Alvaro Sanchez-Gonzaler Nicolas Heess, Jost Tobias Springenberg, Josh Merel, Martin Riedmiller, Raia Hadsell, and Peter Battaglia. Graph networks as learnable physics engines for inference and control. In International Conference on Machine Learning, pp.
PMLR, 2018.

---

Under review as a conference paper at ICLR 2023
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with ing. pp. 1842-18500 PMLR, 2016.
neural networks. In International conference on machine learn-
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 2017.
Gustaf Tegnér, Alfredo Reichlin, Hang Yin, Mårten Björkman, and Danica Kragic. On the subspace structure of gradient-basee baaee meta-learning. arXiv preprint arXiv:2207.0388 2022.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Jixuan Wang, Quan-CCiich Wang, Frank Rudzicz. and Michael Brudno. Grad2task: Improved few- shot classification using gradients for task representation. Advances in Neural Information Processing Systems, 34:6542-6554, 2021.
Yaqing Wang, Quanming Yao. James T Kwok, and Lionel M Ni. Generalizing from few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3):1-34, 2020.
Karl Weiss, Taghi M Khoshgoftaar, and DingDing Wang. A survey of transfer learning. Journal of Big data, 3(1):1 0:11-0022200 2016.
Jun Wu and Jingrui He. unified meta-learning framework for dynamic transfer learning. arXiv preprint arXiv:2207.0177 2770017444 2022.
Zhou Xian, Shamit Lal, Hsiao- Yu Tung, Emmanouil Antonios Platanios, and Katerina Fragkiadaki. Hyperdynamics: Meta-learning object and agent dynamics with hypernetworss. arXiv preprint arXiv:2103.09439, 2021.
Jin Xu, Jean-Francois Ton, Hyunjik Kim, Adam Kosiorek, and Yee Whye Teh. Metafun: Meta- learning with iterative functional updates. In International Conference on Machine Learning, pp- 10617-10627. PMLR, 2020.
Zhenjia Xu, Jiajun Wu, Andy Zeng, Joshua B Tenenbaum, and Shuran Song. Densephysnet: Learn- ing dense physical object representations via multi-step dynamic interactions. arXiv preprint arXiv: /90::/96..00 03853, 2019.
Wenhao Yu, Jie Tan, Karen Liu, and Greg Turk. Preparing for the unknown: Learning a universal policy with online system identification. arXiv preprint arXiv:1702.02455. 2017.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander Smola. Deep sets. Advances in neural information processing systems, 30, 2017.
Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1): 43-76, 2020.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. International Conference on Machine Learning, pp. 7693-7702.
PMLR, 2019.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 IMPLEMENTATION DETAILS
In all of our experiments, we let the implicit representation learner fo be a 3-layer MLP with 40 hidden units. For the non-linear method based on LEO Rusu et (2018), we implement the hyper- network as a 2-layer MLP with 128 hidden units.
For the pooling methods, we let h be an MLP with hidden layers of width 128 and final out- put layer of size k. For the transformer, we first encode each input separately with h. We then implement the query, key and value networks as linear layers before we calculate the output as SOFTMAX(QK' )V. For METAFUN, layer MLP into a 128-dim vector. We then utilize an RBF kernel with temperature = 1.0. The outputs multiplied by the outputs of a network that is implemented another 2-layer MLP. We (Ei= k( x (r( .
encode each point in the support and query through a 2-
employ final network w on the outputs to find y
We use learning-rate of 5e 4 and train for 100 epochs. All results presented are averaged over three different random seeds with the mean and standard deviation calculated on different held-out test sets. For all experiments we use 2-layer MLP for the decoder network MD
A.2 ALGORITHM
Algorithm
Require:
p(T)), P(G)): distributions over tasks, randomly initialize
while not done do
sample task Ty ~ (TFF), ~ p(Tg) sample batch of data-points D, ~ Ty. Dg Tg
(M
According to Equation
4
5
(=) update + & - a VVeLr,
end while
A.3 ADDITIONAL PENDULUM EXPERIMENTS
Model
0.0
== 0.4
a= 1.0
= 2.0
a 3.0
GRADIENT
CCNTEXT
0.33_0.01
0.33+0.02
0.33+0.02
0.34+0.01
0.35+0.01
NONN-LINEAR
0.33
0.32+0.02
0.32+0.02
0.33+0.02
0.38+0.07
-LINEAR
0.32
0.33
0.33
0.35
0.41
POOLING
-MEAN
0.33-0.01
0.33
0.35+0.01
0.42+0.03
0.60+0.14
.MAX
0.31
0.35+0.02
0.40+0.07
0.57+0.22
1.1940.84
-TRANSFORMER
0.30
0.32+0.02
0.34+0.02
0.73
3.45
DEEP KERNEL
1.35
1.36+0.08
1.67+0.06
2.5640.54
3.14
E((441 21
3.23
Table 4: Results for double pendulum experiment
For reference we also compute the average distance between consecutive points. This would corre- spond to always predicting the current state, thus computing B[(x411 2..
13

---

Under review as a conference paper at ICLR 2023
Figure 5: Three examples of the maze environment. The start and goal position are defined by the red and orange marker respectively.
A.4 ANT EXPERIMENT DETAILS
In the environment, the agent consists of a torso and four legs. Each leg is composed of two links (upper-leg, ankle) joint together. generate different tasks, we vary the length of the upper-leg and the ankle for all the legs. To retain symmetry, for each of the four legs we consider the same adjustment and thus leaving us with a two degree-of-freodden change between the tasks. The goal of the agent is to walk as far as possible in the x-dirrccion while maintaining certain stability.
A.5 MAZE EXPERIMENT DETAILS
We define the maze over 2 x N grid. From every other position on the top row of the maze ( positions) we sample an obstacle with probability p 0.5. We then define the obstacles on the bottom row as the complement of the top row. Three examples can be seen in Figure 5
14


Reviews:
[{"review_id": "x6oslm72O-", "reviewer": "Reviewer_VvbW", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to solve support-query distributional shift problem which has not been addressed by the previous meta-learning literatures. Instead of assuming that the same function f is used to sample both support and query set, they assume that different functions f and g are generating each support and query set. And then they propose to learn to map from f to g with simple transformations. The proposed method outperforms the simple baselines over various problems, especially well when the white noises are added.", "summary_of_the_review": "In summary, the paper addresses a very important problem of support-query distributional shift that has not been fully addressed by the previous meta-learning literatures. The method is clear, simple, and effective. I thus recommend acceptance, but for the novelty part I will have a discussion with other reviewers.", "strength_and_weaknesses": "Strength\n- The paper is well written\n- The problem formulation is novel and important\n- The method is simple and intuitive\n- The proposed method outperforms the simple baselines over various problems\n- The proposed method provides a new way to generalize to a distributional shift\n\nWeaknesses\n- As already pointed out in the main paper, the dimensionality of both the parameters of the function f (source) and g (target) are high-dimensional, which may prevent the proposed method from being applied to larger scale problems. As far as I understand, the experiments are all small scale due to this reason.\n- I wonder whether there exists no such literatures solving exactly the same problem. It's quite surprising. Maybe there should be some that I'm not aware of. I will defer this point to the discussion phase with other reviewers.\n- It's unclear why the proposed method should outperform with strong noise level. Could you provide some intuition?\n- Although it is nice that the method provides a way to generalize to a distributional shift, it is only for a single specific distributional shift at a time. For instance, in pendulum experiments it is only generalizable to double pendulum, while we would wish to generalize to any number of pendulums.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "RXGU9hDbDNQ", "reviewer": "Reviewer_8eQb", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The core of the method is its function representation, which takes the expectation of a low-dimensional gradient over the source set. The paper presents three variants for this step. The decoder is a hyper network which maps the function representation to parameters.", "summary_of_the_review": "This paper proposes an interesting idea but its empirical evaluation is not very strong.", "strength_and_weaknesses": "I’m assuming things like theta are also trained end-to-end in equation (8). If so, this should be stated more explicitly.\n\nFor the “context” variant for example, does the learned theta perform well on the source task, or does this network only produce good gradients for task representation while producing meaningless outputs?\n\nAcross all experiments, what is the performance of each method without noise? I think that is important for getting a sense of how severely noisy these settings (e.g. sigma=2.0) are.\n\nExperimental evaluation is somewhat limited.\n\nI think the problem setting is interesting; going beyond the i.i.d. assumption within  the source and query data is a nice direction. Though, I wouldn’t agree that “transfer learning” is the best way to describe it. I think transfer learning generally refers to the setting where you have some amount of information about the target task, whereas in this setting we directly produce a model for T_g before seeing any data from the domain.\n\nThe experiment in Figure 2 is compelling: gradient-based function representation automatically cancels out noise.\n\nPage 4: The language around functionals is confusing. You call f and g functionals, but I think they are standard functions? You also say that T is a map from function to function, but then you do function composition between T and f, which implies that T is a function from Y to Y (where Y is the codomain of f).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "Q4r4vJTbrD", "reviewer": "Reviewer_6CPS", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors proposed a novel model-based meta-learning method, specifically applied to the problem of transfer learning, where support and query data come from two separate (but related) tasks. This formulation allows flexibility in terms of models that can be applied, in that the model used for support examples need not be the same as the one for query examples. This method is based on gradient information on the support loss, and conditions the parameters of the model on the query examples. This method has been compared to other model-based meta-learning methods, on various environments.", "summary_of_the_review": "There are crucially missing details in the current submission regarding the proposed method, and some confusion regarding how to frame this method. That's why I am currently recommending rejection, but I am willing to increase my score if these concerns are properly addressed.", "strength_and_weaknesses": "**Strengths**: The presentation of the theory is for the most part precise. The empirical evaluation is extensive, over multiple environments with varying degrees of complexity. The comparison with standard model-based meta-learning methods as the encoder $M^{E}$ is appreciated.\n\n**Weakness**:\n 1. What is the connection between your proposed method (as introduced in Section 4.1) and GBML, as presented in Section 3.2? It seems that the proposed method does not use any adaptation through and optimization procedure as in GBML, but is closer to model-based meta-learning, where the encoder $M^{E}$ happens to use gradient information. As such, it is incorrect to call the proposed method GBML (as in Section 7, \"*a family of method at the intersection between GBML and model-based techniques*\", when the proposed method is purely model-based).\n 2. Although most of the theoretical part is precise, it is crucially missing some details when it comes to the proposed method. In particular, there is no detail as to the point at which the gradient information used in $M^{E}$ is evaluated. For example in Equation 4, $\\mathbb{E}[\\nabla_{z}L(f(x, z), y)]$, at what point $z$ is this gradient evaluated? Is $z$ a learned meta-parameter? This is crucially missing from the description of the method; for example, if $z$ is meta-learned, then you should explicitly write down the meta-learning objective in the paper.\n 3. Although the experiment on next-state prediction for pendulum/double pendulum is interesting, in practice what is the temporal resolution of the data? I feels like this problem might be too simple if the temporal resolution is very high.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "1bNX8f9fb8z", "reviewer": "Reviewer_9QJw", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper provides an meta-learning styled transfer learning method that essentially applies meta-learning to the case where supervised learning tasks vary w.r.t. their underlying functions. Experiments are performed over different problems (including regression, learning dynamics, and imitation learning). ", "summary_of_the_review": "My score is mainly based on some points made in the paper that seem dubious to me, and raise concerns regarding the correctness of this work. These are listed above. Perhaps I have misunderstood the paper, and thus look forward to the discussion period. ", "strength_and_weaknesses": "### Strengths\n-  \n\n### Weaknesses\nSome things aren't clear to me:\n- Section 2: Why can't transfer learning be considered to perform general adaptation? What is your definition of general adaptation?\n- In Section 4, you mention that support and query sets are from the same task? My understanding is that $T_f$ contains all possible pairs of $x \\in X$ and $y \\in Y$ such that $f(x) = y$. So then, are you claiming that the datasets used in general meta-learning are simply just different sample sets of $T_f$, under the same $f$? I don't think this is true. My understanding is that $f$ can change, within an assumed (function) space.\n- The double pendulum is a chaotic system. Transferring from a single pendulum here, although makes intuitive sense, seems to me could be difficult problem. A discussion regarding how this is addressed, either implicitly, or explicitly would have been nice to see. \n- Are the experiments only comparing against itself? Discussing why this is the case (other methods aren't comparable?) is needed. Could you have checked against vanilla meta-learning as a baseline?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject