Interpretable Meta-Learning of Physical Systems
Authors: Matthieu Blanke
Marc Lelarge
Venue: ICLR 2024 poster (2024)
Keywords: meta-learning
physical systems
multi-task learning
interpretable deep learning
identifiability
electrostatics
robotics
control
reinforcement learning
scientific discovery
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We propose a new multi-environment meta-learning architecture for physical systems called CAMEL, that learns and generalizes at minimal cost and with interpretable weights.

Abstract: Machine learning methods can be a valuable aid in the scientific process, but they need to face challenging settings where data come from inhomogeneous experimental conditions. Recent meta-learning methods have made significant progress in multi-task learning, but they rely on black-box neural networks, resulting in high computational costs and limited interpretability. We introduce CAMEL, a new meta-learning architecture capable of learning efficiently from multiple environments, with an affine structure with respect to the learning task. We prove that CAMEL can identify the physical parameters of the system, enabling interpreable learning. We demonstrate the competitive generalization performance and the low computational cost of our method by comparing it to state-of-the-art algorithms on physical systems, ranging from toy models to complex, non-analytical systems. The interpretability of our method is illustrated with original applications to parameter identification and to adaptive control and system identification.

Published as a conference paper at ICLR 2024
INTERPRETABLE META-LEARNING OF PHYSICAL SYSTEMS
Matthieu Blanke
Marc Lelarge
Inria Paris, DI ENS, PSL Research University matthieu blanke iiriia.fr.....
Inria Paris, DI ENS, PSL Research University mare. Lnraggeiriaa
ABSTRACT
Machine learning methods can be a valuable aid in the scientific process, but they need face challenging settings where data come from inhomogeneous exper- imental conditions. Recently, meta-learning approaches have made sigmificant progress in multi-task learning, but they rely on black-box neural networks, re- sulting in high computational costs and limited interpretability. Leveraging the structure the learning problem, we argue that entiromennion generalization can be achieved using simpler learning model, with an affine structure with re- spect to the learning task. Crucially, we prove that this architecture can identify the physical parameters of the system, enabling interpretable learning. We demon- strate the competitive generalization performance and the low computational cost of our method by comparing it to state-of ffate-ffthee-as -he- algorithms on physical systems, ranging from toy models to complex, non-analytical systems. The interpretability of our method is illustrated with original applications to parameter identification and to adaptive control.
INTRODUCTION
Learning physical systems an essential application of artificial intelligence that can unlock signif- icant technological and societal progress. Physical systems are inherently complex, making them difficult learn Karniadakis et al. (202T). A particularly challenging and common scenario is multi- environment learning, where observations of physical system are collected under inhomogeneous experimental conditions Caruana (1997). such cases, the scarcity training data necessitates the development of robust learning algorithms that efficiently handle environmental changes and make use of all available data.
This multi-environment learning problem falls within the framework of multi-task learning, which has been widely studied the field of statistics since the 1990s (Caruana 1997). The aim is to exploit task diversity to learn shared representation of the data and thus improve generalization. With the rise of deep learning, several meta-learning approaches have attempted in recent years to incorporate multi-task generalization into gradient-based training of deep neural networks. In the seminal paper by Finn et al. (2017). and several variants that followed (Zintgraf et al. 2019 Raghu al., 2020), this done by integrating an inner gradient loop the training process. Alterna- lively, Bertinetto et al. (2019) proposed adapting the weights using a closed-form solver. As far as physical systems are concerned, the majority of the proposed methods have focused on specific architectures oriented towards trajectory prediction (Wang et al. 2022a Kirchmeyer et al., (022).
When learning physical system from data, a critical yet often overlooked challenge is model in- terpretability (Lipton) 2018 Grojean et al. (022). Interpreting the learned parameters in terms of the system's 's physical quantities crucial to making the model more explainable, allowing for sci- entific discovery and downstream model-
applications such control. The above approaches benefit from the expressiveness of deep learning, but are costly in terms of computational time, both for learning and for inference. Furthermore, the complexity and the black-bor nature of neural net- works hinder the interpretability of the learned parameters, even when the physical system is linearly parametrized.
Recently, Wang et al. (202I) showed theoretically that the learning capabilities of gradient- meta-learning algorithms could be matched by the simpler architecture of multi-task representation

---

Published as a conference paper at ICLR 2024
learning with hard parameter sharing, where the heads of neural network trained to adapt to multiple tasks (Caruanal 1997; Ruder, 2017). They also demonstrated empirically that this archi- madieett-based meta-leaming algorithms for few-shot
tecture is competitive against ofateof--theart image classification. We propose to use multi-task representation learning for physical systems, and show how it can bridge the gap between the power of neural networks and the interpretability of the model, with minimal computational costs.
Contributions In this work, we study the problem of multi-environmenn learning of physical sys- tems. We model the variability of physical systems with a multi-task representation learning archi- tecture that is affine in task-specific parameters. By exploiting the structure of the learning problem, we show how this architecture lends itself to multi-environment generalization, with considerably lower cost than complex meta-lcarning methods. Additionally, we show that enables identification of physical parameters for linearly parametrized systems, and local identification for arbitrary sys- tems. Our method's 's generalization abilities and computational speed are experimentally validated on various physical systems and compared with the state of the art. The interpretability of our model illustrated by applications to parameter identification and to adaptive control.
LEARNING FROM MULTIPLEE PHYSICAL ENVIRONMENTS
In this section, we present the problem of multi-task learning as it occurs in the physical sciences and we summarize how can be tackled with deep learning meta-learning framework.
2.1 THE VARIABILITY PHYSICAL SYSTEMS
In general, physical system is not fixed from one interaction to the next, as experimental conditions vary, whether in a controlled or uncontrolled way. From learning perspective, we assume meta- dataset D := - D, composed of T datasets, each dataset gathering observations of the physical system under specific experimental conditions. The goal to task changes, in the sense that when presented a new task, it can learn the underlying function from a few samples (Hospedales et al. >. Note that in practice the number of tasks typically very limited, owing to the high cost of running physical experiments.
learn predictor from D that robust
For simplicity, we assume classical supervised regression setting where D. := y/ JisieN. and the goal is to learn x + y predictor, although the approaches presented generalize to other settings such as trajectory prediction of dynamical systems. We discuss two physical examples illustrating the need for multi-task learning algorithms, with different degrees of complexity.
Example (Actuated pendulum). We begin with the pendulum, one of physics' most famous toy systems. Denoting its inertia and its mass by I and m and the applied torque by u, the angle obeys
Iq + mg sin For example, we may want to learn the action y = u as a function of the coordinates x = (g. 4. Q) In a data-driven framework, the trajectories collected may show variations in the pendulum parameters: the same equation (2.1) holds true, albeit with different parameters m and I.
(2.1)
A more complex, non-analytical example is that of learning the solution to partial differential equa- tion, which rarely known in closed form and varies strongly according to the boundary conditions.
Example (Electrostatic potential). The electrostatic potential in a space Ω devoid of charges solves I Laplace's equation, with boundary conditions
Ay on Ω y(z) = b(z) on in
(2.2)
A robust data-driven solver should be able to generalize to (at least small) changes of a and
2.2 OVERVIEW OF MULTI-ENVIRONMENT DEEP LEARNING
Multi-task statistical learning has long history, and several approaches to this problem have been proposed in the statistics community (Caruana 1997). We will focus on the meta-learning paradigm (Hospedales et al. 202I), which has recently gained considerable importance and whose application to neural nets looks promising given the complexity of physical systems. We describe the generic structure of meta-leamiig algorithms for multi-task generalization.
2

---

Published as a conference paper at ICLR 2024
Learning model Given the learning capabilities of neural networks, incorporating multi-task gen- eralization into their gradient descent training algorithms is a major challenge. Since the seminal paper by/Fin et al. (2017), several algorithms have been proposed for this purpose, with the common idea of finding a map adapting the weights of the neural network according to task data. A conve- nient point of view is to introduce a two-fold parametrization of meta-model Far:; v.), with task-agnostic parameter vector 0 E IR" and task-specific weights (also called learning contexts). For each task 6, the task-specific weight is computed based on some trainable meta-parameters and the task data currently being processed as Wi = A(T, Dz), according to an adaptation rule A that is differentiable with respect æ. The meta-parsmeters are trained to minimize the meta-loss function aggregated over the tasks, as we will see below.
We provide examples of recent architectures in Table [ In MAMI. (Finn al. 2017). the meta- parameter is simply 0 and the adaptation rule is computed as gradient step in the direction of the task-specific loss improvement, an inner gradient loop. In CoDA (Kirchmeyer et al.) 2022). the meta-parameter has dimension growing with the number of tasks & and the adaptation rule computed directly from the meeta-arrameters with task-specific low-dimensional context vec- tors & E 2 and linear hypernetwork e R Variants of MAML, CAVIA (Zintgraf al. 2019) and ANIL (Raghu et al. 2020), fit into this scheme as well and correspond to the restriction of the adaptation inner gradient loop to predetermined set of the network's 's weights. This framework also encompasses the CAMEL algorithm, which we introduce in Section
Meta-training
The training process is summarized in Algorithm meta-learner computes a task-specific version of the model from the task dataset Dz, defin- ing fo T) = Flx; 0, ATT,, D.)). The error on the dataset D, is measured by the task-specific loss
For each task the
Σ 1 2 (F(=; 2,yED.
(D;;0,w)
(2.3)
Parameters are trained by gradient descent in order to minimize the regularized meta-loss defined as the aggregation of L, and regularization term R(T):
L(x) = Σ (DDn0, (x) + R(T).
(2.4)
Table 1: Structure of various meta-learning models.
Algorithm Gradient-based meta-training
Here h(x; 0) E R and v(z; 0) E R' denote arbitrary parametric models, such as neural networks; "order" stands for differentiation order.
input meta-model F(x; 0, w), adaptation
rule A, initial parameeasssssrss t, learning
rate 17. task datasets Di.... Dr
output learned meta- veta-parameters π while not converged do for tasks < = T do
MAML
CoDA
CAMEL
9
O,
(EF)
0, (w)) ptrxT
dim(()
ptpx + xT
compute 0 from adapt w. (TT) A(m, D.)
dim(w)
p
A(T, D.)
Vo
O&
We w (((;()))
compute ((D;; 0, (T())) end for
F(z; 0,wr)
xz + w)
training
2
order
compute I(TT), as in (2.4 update π + π 1 L(
adaptation
0
order
end while
Test-time adaptation At test time, the trained meta-model is presented with dataset Dr+ 1 con- sisting of few samples (or shots) from a new task. Using this adaptation data and the learned meta- parameters म, the task-agnostic component 0 of the meta-model frozen, and the task-specific com- ponent is tuned (possibly in a constrained set) by minimizing the prediction error on the adaptation dataset:
WT+1 E argmin (Dr+110,,))
(2.5)
In all the above approaches, this minimization is performed by gradient descent. The resulting adapted predictor is defined as F(x; 0, W1+1). and is evaluated by its performance on a separated test set from task T 1, averaged over the task distribution.
3

---

Published as a conference paper at ICLR 2024
3 CONTEXT-AFFINT MULTI-ENVIRONMENT LEARNING
Physical systems often have a particular structure in the form of mathematical models and equa- tions. The general idea behind model- based machine learning is to exploit the available structure to increase learning performance and minimize computational costs (Kariiadaiis et al. (20TT). With this in mind, we adopt in this section simpler architecture than those shown above, and show how it lends itself particularly well to learning physical systems.
Problem structure We note that many equations in physics exhibit an affine task dependence, since the varying physical parameters often are linear coefficients (as we see in Example and we shall further explain in Section[4 4). By incorporating this same structure and hence mimicking phys- ical equations, the model should be well-suited for learning them and for interpreting the physical parameters. Following these intuitions, we propose to learn multi-environment physical systems with affine task-specific context parameters.
Definition (Context-affine multi-task learning). The prediction modeled as an affine function of low-dimensional task-specific weights E IK' with task-agnostic feature map vz; 0) E R" and task-agnostic bias e(2; 0) E R:
F(x; w) c(x;; w v(x;; 0).
(3.1)
The dimension of the task weight must be chosen carefully. It must be larger than the estimated number of physical parameters varying from task to task but smaller than the number of training tasks, so to observe the function u projected over sufficient number of directions. During train- ing, the task-specific weights are directly trained as meta meta-paramett parameters along with the shared parame- ter vector: (0, w1 ... ,.,,,,))) and Wh A(T, D2) we The meta-parameters are jointly trained by gradient descent as in Algorithm| reduces ordinary least squares.
1
At test time, the minimization problem of adaptation (2.5)
The architecture introduced in Definition is equivalent to multi-task representation learning with hard parameter sharing Ruder (2017) and is proposed as a meta-learning algorithm in (Wang et al.] 2 We will refer it in physical system framework as Context-Affine Envii-Enviroommnt Learning (CAMEL). In this work, we show that CAMEL is particularly relevant for learning physical systems. Table I compares CAMEL with the meta-learning algorithms described above.
Computational benefits As the task weights (w))T= are kept in memory during training instead of being computed in an inner loop, CAMEL can be trained at minimal computational cost. In particular, it does not need to compute Hessian-vector products gradients through matrix inversions as in (Bertinetto et al., 2019). Adaptation at test time is also computationally inexpensive since ordinary least squares guarantees a unique solution in closed form, as long as the number of samples exceeds the dimension of the task weight. For real- time applications, the online least-squares formula (Kushner & Yin 2003) ensures adaptation with minimal memory and compute requirements, whereas gradient-based adaptation (as in CoDA or MAML) can excessively slow.
in MAML,, to propagate
Applicability The meta-learning models described in Section 2.2 seek to learn multi-task data
from complex parametric model (typically neural network), making the structural assumption that the weights vary slightly around central value in parameter space: fe(; T)) = h(x; 9, + 80,), with ||ā0|| << ||o|]. Extending this reasoning, the model should be close to its linear approximation:
h(:; 0o + 50,) h(x;00) + 50, Tv((:; 00),
(3.2)
where we observe that the output an affine function of the task-specific component 80,- We believe that (3.2) explains the observation that MAML mainly adapts the last layer of the neural network (Raghu et al., 2020). In Definition [ v and c are arbitrary parametric models, which can be as complex as a deep neural network and are trained to learn representation that is linear in the task weights. Following (3.2). we expect CAMEL's expressivity to be of the same order as that of more complex architectures, with c(z; 8), we and v(z; 8) playing the roles of h(æ; ou), 59, and Vh(x; [ ff) respectively. Another key advantage of CAMEL is the interpretability of the model, which we describe next.
4

---

Published as a conference paper at ICLR 2024
4 INTERPRETABILITY AND SYSTEM IDENTIFICATION
The observations a physical system are often known to depend on certain well-identified physical quantities that may be of critical importance in the scientific process. When modeling the system in a data-driven approach, it is desirable for the trained model parameters to be interpretable in terms of these physical quantities (Karniadakis al. 202T). thus ensuring controlled and explainable learn- ing (Linardatos et al. 2222)). We here focus on the identification of task-varying physical parameters, which raises the question of the identifiability of the learned task-specific weights. System identifi- cation and model identifiability are key issues when learning a system (Ljung) 1998). Although deep neural networks are becoming increasingly popular for modeling physical systems, their complex structure makes them impractical for parameter identification in general (Nelles 2001).
mathematical terms, the observed output considered as an unknown function y( of the input and a physical context vector yp E R", gathering the pa- rameters of the system. In our multi-environment setting, each task defined by a vector yr as (xr) (t)) At test time, new environment corresponds to an unknown underlying physical context 9r--1 While adaptation consists in minimizing the prediction error on the data as in (2.5). the interpretation goes further and seeks to identify (P++1- This means mapping the learned task-specific weights w to the physical contexts yp, i.c. learning an estimator & w + ip using the training data and the trained model. Assuming that the physical parameters of the training data } are known, this can be viewed as a regression problem with T samples, where & is trained predict sp; from weights we, learned the training meta-dataset.
Physical context identification
4.1 LINEARLY PARAMETRIZED SYSTEMS
We are primarily interested in the case where the physical parameters are known to intervene linearly in the system equation, as
y<:: = x(x) + o vT),,
v(z) E R".
(4.1)
This class of systems is of crucial importance: although simple, it covers a large number of problems of interest, as the following examples illustrate. Furthermore, it can apply locally to more general system, as we shall see later.
Example 3 (Electric point charges). Point charges are particular case of Example 2 with point boundary conditions. proportional to the charges be computed using Coulomb's law and is proportional to these charges: y(æ; sp) = sp (xr),, with v(x) x (1/1|z 2 Although the solution is known in closed form, this example can illustrate more complex problems where an analytical solution is out of reach (and hence v is un- known) but the linear dependence on certain well-identified parameters is postulated or known.
pf()). The resulting field can
Example 4 (Inverse dynamics in robotics). One application where our model is particularly well suited in robotics is inverse dynamics: it turns out that the Euler-Lagrange formulation for the rigid body dynamics is always linear with respect to the system's dynamic parameters (Nguyen- Tuonn- & Peters. 2010), and hence takes the form of (4.1). A simple, yet illustrative system with this structure the actuated pendulum 2.11. where it is clear that the equation is linear in the inertial parameters I and m. The inverse dynamics equation can be used for trajectory tracking (Spong et al. 2020), as predicts from target trajectory {4( {)). We provide more details in Appendix B.3
4.2 LOCALLY LINEAR PHYSICAL CONTEXTS
In the absence of prior knowledge about the system under study, the most reasonable structural as- sumption for multi-task data is to postulate small variations in the system parameter: yp (P + dip. The learned function can then be expanded and found to be locally linear in physical contexts:
y( sp) y( + 8Gp Vy(x; (Po),
(4.2)
which has the form with <(x) = أ(: (P)) and v(z) Vy(z; po).
Example 5 (Identification of boundary perturbations). For a general boundary value problem such as (.. we may assume that the boundary conditions @g({)) b(r, (p) vary smoothly according to parameters ip (such as angles or displacements). If these variations are small and the problem is sufficiently regular, the resulting solution yaz, sp) can be reasonably well approximated by (4.2)
5

---

Published as a conference paper at ICLR 2024
4.3 PARAMETER IDENTIFICATION WITH CAMEL
We now study the problem of system identification under the assumption of parameter linearity 4.1 using the CAMEL metamodel (3.1). We study the identifiability of the model and therefore investi- gate the vanishing training loss limit, with c = K = 0 for simplicity, yielding
Wt
T v((r(") :p: Tp:TTT vv((())) for all 1 1 T, 1 S N..
(4.3)
Identifiability Posed as it is, we can easily see that the physical parameters is are not directly identifiable. Indeed, for any P (RR,, the weights w and the feature map e produce the same data as the weights w' == P and the feature map v' = P- since w v = w PP-1v. This problem is related to that of identification in matrix factorization (see for example Fu et al. (2018). ). Now that we have recognized this symmetry of the problem, we can ask whether it characterizes the solutions found by CAMEL. The following result provides positive answer.
Proposition 1. Assume that the training points are uniform across tasks: x;") = x00,, and N₂ = N for all 1 < < T and I i < N, with n S r < N, T. Assume that both sets (u(x())) and {pt span R" In the limit of vanishing training loss L(xT) = the trained mneaa-paraettes recover the parameters of the system up to linear transform: there exist P. P.Q E Raxr such that yo: = Pw; for all training task i and ((x(())) = Qu(x()) for all 1 < i S N. Additionally, QP In-
O. Proposition 1 shows that CAMEL learns a meaningful representation of the system's 's features instead of overfitting the examples from the training tasks. Remarkably, the relationship between the learned weights and the system param- eters is linear and can be estimated using ordinary least squares:
A proof is provided in Appendix A along with the
T Σ IP - Pell
(ww) Pw. P E argmin
(4.4)
PEREXT 2
For black- box meta-learning architectures, exhibiting the symmetries in model parameters and com- puting identification map seems out of reach, as the number of available tasks T can be very limited in practice (Pourzanjani et al. 2017).
5 EXPERIMENTING ON PHYSICAL SYSTEMS
The architecture that we have presented expected to adapt efficiently to the prediction of new environments, and identify (locally globally) their physical parameters, shown in Section 4 In this section, we validate these statements experimentally on various physical systems: Sections 5.1 and 5.2 deal with systems with linear parameters (as in (4.1). which evaluate the inter- pretability of the algorithms. We then examine non-analytical, general system in Section 5.3 We compare the performances of CAMEL with state-of-the-art meta-learning algorithms. Our code and demonstration material are available at aottttps: /tttps://githut com MB 229 hes//meta--e latttaaeerninnn
Baselines We have implemented the MAML algorithm of Finn et al. (2017). and its ANIL vari- ant (Raghu et al., 2020). which is computationally lighter and more suitable for learning linearly parametrized systems (according to observation (3.2) We have also adapted the & -CDDA architec- ture of Kirchmeyer et al. (2022) for supervised learning (originally designed for time series predic- tion). In all our experiments, the different meta-models share the same underlying neural network architecture, with the last layer of size r 2 tlim(pp) Additional details can be found in Appendix B
The linear regressor computed for CAMEL in (4.4) is computed after training for all architectures with their trained weights Wt, and is available at test time for identification.
5.1 INTERPRETABLE LEARNING OF AN ELECTRIC POINT CHARGE SYSTEM
As a first illustration of multi-envirrnment learning, we are interested in a data-driven approach to electrostatics, where the experimenter has no knowledge of the theoretical laws (Maxwell's equations, as in Example 2 of the system under study. The electrostatic potential is measured various points in space, under different experimental conditions. The observations collected are then used to train a meta-learning model to predict the electrostatic field from new exper- iments, based on very limited data. We start with the toy system described in Example 3

---

Published as a conference paper at ICLR 2024
target
ANIL
CAMEL
target
CoDA
CAMEL
J2
x
32
Figure 1:
Few-shot adaptation on two out-of-domain environments of the point charge system in dipolar setting (left) and the capacitor (right). The adaptation points are represented by the x sym- bols. The vector fields are derived from the learned potential fields using automatic differentiation.
which provides qualitative illustration of the behavior of various learn- ing algorithms: point charges placed in the plane at fixed locations. This experiment is repeated with varying charges E TR33
identification error
10-1
10-2
Results For this system with linear physical parameters, CAMEL out- performs other baselines and can predict the electrostatic field with few shots, as shown in Figure | and Table 2 (5-shot adaptation). Figure 2 shows the identification error
109
10-
T
ANIL
CoDA
CAMEL
30 random test environments with standard deviations, as a function of the number of training tasks. Thanks the sample complexity of linear regression, CAMEL accurately identi- fies system charges, achieving less than 1% relative error with 10 training tasks.
Figure Average rel- ative error for the point charge identification.
5.2 MULTI-TASK REINFORCEMENT LEARNING AND ONLINE SYSTEM IDENTIFICATION
Another scientific field in which our theoretical framework can be applied is multi-task reinforce- ment learning, which control policy learned using data from multiple environments one system /Vithayathil Varghese & Mahmoud] 2020). We saw in Example 4 that robot joints obey the inverse dynamics equation, which turns out to be linear the robot's s inertial parameters. Conse- quently, our architecture lends itself well to the statistical learning of this equation from multiple environment data, as well as to the identification of the dynamic parameters. We may then exploit the learned model of the dynamics perform adaptive inverse dynamics control (see Appendix B.4) of robots with unknown parameters, and identify the parameters simultaneously.
Systems We experiment with systems of increasing complexity, starting with 2D simulated sys- tems: cartpole and acrobot. To make them more realistic, we add friction in their dynamics. The an- alytical equation (4) is hence inaccurate, which motivates the use of data-driven learning method. We then experiment don't know the true inverse dynamics function and the wheel torque is learned from the ground position and the joint angles.
the simulated delegree-of-rreedee robot Upkie (Figure 3). for which we
Experimental setup Learning algorithms are trained on trajectories (a more challenging setting than uniformly spaced data) obtained from multiple system environments. At test time, ronment is instantiated and the model is adapted from a trajectory of few observations. The resulting adapted model is then used to predict control values for the rest of the trajectory. For the carptole and the robot arm, the predicted values are used to track reference trajectory using inverse dy- namics control. For Upkie, we could directly use the predicted controls for actuation, but compare the open-loop predictions with the executed control law. The target motions are swing-up trajectories for the cartpole and the arm, and 0.5m displacement for Upkie. Since Upkie a very unstable system, it is controlled in a 200Hz model predictive control loop (Rawlings 2000).
envi-
Online adaptive control We also investigate a challenging vire-ng dynamics setting where the inertial parameters of the system change abruptly a given time. This scenario is very common in real life and requires the development of control algorithms robust to these changes and fast enough to be adaptive (Aström & Wittenmark 2013). In our case, double the mass of the cart in

---

Published as a conference paper at ICLR 2024
cartpole
cartpole
Upkie
tip height
adaptation
Up position
position
0.5
10-
0.0
Gargee
torque
arm
0
unialyrae
tip height
adaptation
mass
3.0
-0.3
mass
CAMELL
1.5
0
time 100
time
1000
time
3000
Figure 4: Tracking of reference trajectory using the learned inverse dynamics controller. Left. 50-shot adaptation. Center and right. The model and the controller are adapted online.
the cartpole system, and we quadruple the mass of Upkie's torso. The learning models adapt their task weights online and adjust their control prediction. an application to parameter identification, also compute the estimated values of the varying parameter over time.
Results The 100-shot adaptation error of the control values is reported in Table The trajectories obtained with inverse dynamics control adapted from 50 shots are plotted Figure 4 for CAMEL and for the best-performing baseline, ANIL, along with the analytical solution. Only CAMEL adapts well enough to track the target trajectory. The analytic solution underestimates the control does not account for friction, resulting in inaccurate tracking. In the adaptive control setting, the variation the mass of cart leads deviation from the target trajectory but CAMEL is able to adapt quickly to the new environment and identifies the new mass, unlike ANIL. Experimentation on Upkie shows that the computational time adaptation can be crucial, as found that the gradient-based adaptation of ANIL and CoDA Upkie.
Figure 3:
was too slow to run the 200Hz model predictive control loop. On the other hand, CAMEL's gradient-free frelie adaptation and interpretability allow it to track and identify changes in system dynam- ics, and to correctly predict the stabilizing control law.
5.3 BEYOND CONTEXT-LINEAR SYSTEMS
In order evaluate method general sys- tems with no known parametric structure, we consider the following non-analytical electro- static problem of the form shown in Example 2 The field is created by capacitor formed by electrodes that are not exactly parallel. The vari- ability the different experiments stems from the misalignment dp E IR, in angle and posi- tion, of the upper electrode, We apply the same methodology as described in Section 5.1 The whole muuti-ennvronmeet learning experiment is repeated several times with varying magnitudes of misalignment, by replacing 8yp with e cup for different values of E [0, This allows us to move gradually from local perturbations
CAMEL target
identification
10
10-2
0.5
dep
ANIL
CaDA
CAMEL
Figure 5: Adaptation and relative identification error for the e-capacitor, with increasing 8.
when e & (as in Example 5) to arbitrary variations in the environment.
Results The 40-shot adaptation error for the e-capacitor reported Table 2 with perturbation of full magnitude best performing baseline, CoDA, for & = 0.2 Figure T When the system parameters are fully nonlinear, CAMEL and the baselines perform similarly. but CAMEL is much faster. In the second case, CAMEL outperforms them by an order of magnitude and accurately predicts the electrostatic field, whereas CoDA's exhibits lower precision. Predictions and average identification error (with
and with 0.1. We also show the 5-shot adaptation of CAMEL and the
8

---

Published as a conference paper at ICLR 2024
standard deviations) are plotted as a function of in Figure 5 perturbation is well identified, enabling a zero-shot adaptation.
For small E, the system parameter
Table 2:
Average adaptation mean squared error (left) and computational time (right).
System
Charges
Capacitor
e-Capacitor
Cartpole
Arm
Upkie
Training
Adaptation
MAMI.
1.6F-1
N/A
N/A
1.8F0
8.1F-1
1.5B-2
ANIL
9.26-4
3.6F-2
1.1
2.5F-2
7.5k-1
1.9F-2
10
3
CoDA
8.2F-2
2.6F-2
1.0F-3
8.1E-1
9.3F-
2.1R-2
2
8
R2-D2
1.2F-4
3.1
4.2E-4
8.5F-3
3.5B-
2.3B-2
CAMEL
1.0E-4
2.6B-2
1.98-4
3.1B-3
2.4g-1
8.2b-3
RELATED WORK
Multi-task meta-learning Meta-learning algorithms for multi-task generalization have gained popularity (Hospedales et al.] (021), with the MAML algorithm of |Finn et al. (2017) playing fundamental role this area. Based on the same principle, the variants ANIL (Raghu et al. 2020) and CAVIA (Zintgraf et al. 2019) have been proposed to mitigate training costs and reduce over- fitting. Interpretability is addressed in the latter work. using a large number of training tasks. In different line of work. Bertinetto et al. (2019) proposed the R2-D2 architecture where the heads of the network are adapted using the closed-form formula of Ridge regression. The similarities learning
between multi-task representation learning and gradient-basee 2021] from a theoretical point of view, in the limit of a large number of tasks. Unlike our method, the approaches above rely on the assumption that the number of training tasks large (in few-shot image classification for example, where it can be in the millions (Wang et al. 2021 Hospedales
studied in (Wang al.,
al. 202I)), while is typically very limited for physical systems.
Meta-learning physical systems Meta-learning has been applied to multi-enviirnment data for physical systems, with a focus on dynamical systems, where the target function is the flow of a differential equation. Recent algorithms include LEADS (Yin et al. 2021), I, in which the task depen- dence is additive in the output space and CoDA (Kirchmeyer et al. (2022). where parameter iden- tification is addressed briefly, but under strong assumptions of input linearity. [Wann et al. (2022b) propose prssical-context-beee broader point of view, the interpretability of the statistical model can be imposed by adding physical constraints to the loss function (Raissi et al. 2019).
learning, but context supervision is required for training. From a
Multi-task reinforcement learning Meta-learning has given rise to a number of fruitful new ap- proaches in the field of reinforcement learning. [Sodhani et al. (202T) and Clavera et al. (2019) propose multi-task deep learning algorithms, but no structure is assumed on the dynamics and the learned weights can be interpreted only statistically, in the parameter space of a large black-box neural network. Multi-task learning of inverse dynamics with varying inertial parameters is studied in (Williams al., 2008) using Gaussian processes, but parameter identification is not addressed.
CONCLUSION
We introduced CAMEL, a simple multi-task learning algorithm designed for multi-envirrnment learning of physical systems. For general and complex physical systems, we demonstrated that our method performs as well as the -tate-of-the-art, at a much lower computational cost. Moreover, when the learned system exhibits a linear structure in its physical parameters, our architecture is particularly effective, and enables the identification of these parameters with little supervision, inde- pendently of training. The identifiability conditions found in Proposition]] D are not very restrictive, and the effectiveness of the linear identification map is demonstrated in our experiments. We proposed a particular application in the field of robotics where our data-driven method enables concurrent adaptive control and system identification. We believe that enforcing more physical structure in the meta-model, using for example Lagrangian neural networks (Lutter et al. 2019). can improve its sample efficiency and extend its applicability to more complex robots.
While we focused on classical regression tasks, our framework can be generalized to predict dynam- ical systems by combining it with a differentiable solver (Chen et al. 2018). Another interesting avenue for future research the use of active learning, to make the most at out the available training resource and enhance the efficiency of multi-task learning for static and dynamic systems (Wang al. 2023 Blanke& Lelarge 2023).

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was partially supported by the French government under management of Agence Na- tionale de la Recherche as part of the "Investissements d' a'vvenir" program, reference ANR19-P3IA- 0001 (PRAIRIE 3IA Institute).
REFERENCES
Karl Áström and Björn Wittenmark. Adaptive control. Courier Corporation, 2013.
Bertinetto, J Henriques, P Torr, and A Vedaldi. Meta-learning with differentiable closed-form solvers. In International Conference on Learning Representations (ICLR), 2019. International Conference on Learning Representations, 2019.
Matthieu Blanke and Marc Lelarge. FLEX:
an adaptive exploration algorithm for nonlinear systems. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.). Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 2577-2591, PMLR, 23-29 Jul 2023. URL SeLhhtps:: httt::://proceed.nng mlr pttp:./...r./////////aaaaaaa.a..
html
Rich Caruana. Multitask learning. Machine learning, 28:41-75, 1997.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Ignasi Clavera, Anusha Nagabandi, Simin Liu, Ronald S. Fearing, Picter Abbeel, Sergey Levine, and Chelsea Finn. Learning to adapt in dynamic, real-world environments through meta- reinforcement learning. In International Conference on Learning Representations, 2019. URL
net worumrodd
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-1135, PMLR, 2017.
Xiao Fu, Kejun Huang, and Nicholas D Sidiropoulos. On identifiability of nonnegative matrix factorization. IEEE Signal Processing Letters, 2583):328-332, 2018.
Christophe Grojean, Ayan Paul, Zhuoni Qian, and Inga Strümke. Lessons on interpretable machine learning from particle physics. Nature Reviews Physics, 4(5):284 2022.
Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pallern analysis and machine intelligence, 44(9): 5149-5169, 2021.
George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-intfee informed machine learning. Nature Reviews Physics, 3(6):422 440, 2021.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR (Poster), 2015. URL hhtpp://blly uni ani-trier..e http-//trer..dd/////c/ooo//////a
html # #KingmaB14
Matthieu Kirchmeyer, Yuan Yin, Jeremie Dona, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing to new physical systems via context-informed dynamics model. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference Proceedings of Machine Learning Research, pp. 11283-11301. PMLR, 17-23 Jul 2022. URL. www.mmrr...rs....mmmmmmmm v1 62 ki rcw.eyer222...mmmmmmmmmmmmmmm html
Machine Learning, volume 162 of
https:://proces
Martin Kretzschmar. Particle motion in penning trap. European Journal of Physics, 12(5):240,
1991.
H. Kushner and G.G. Yin. Stochastic Approximattin and Recursive Algorithms and Applications.
Stochastic Modelling and Applied Probability, Springer New York, 2003, ISBN 9780387008943. URL htttpt::: /bookks....//////////..////.kk goog le f fr/books7id= hwt.://toddooottttteeeee
10

---

Published as a conference paper at ICLR 2024
Pantelis Linardatos, Vasilis Papastefanopoulos, and Sotiris Kotsiantis. Explainable ai: A review of machine learning interpretability methods. Entropy, 23(1). 2021. ISSN 1099-4300. doi: 10.3390/623010018. URL/hztps: www wwppi. com 1099 40099433002331/138
Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of inter- pretability is both important and slippery. Queue, 16(3):31-57, 2018.
2018.
Lennart Ljung. System identification. In Signal analysis and prediction, pp. 163-173, Springer, 1998.
Michael Lutter, Christian Ritter, and Jan Peters. Deep lagrangian networks: Using physics model prior for deep learning, arXiv preprint 1907::900.0444 2019.
O. Nelles. Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models, Engineering online library. Springer, 2001. ISBN 9783540673699, URL https:
7/books google f http//ooooo....ooggggeeeeeeeeeee
Duy Nguyen- and Jan Peters. Using model knowledge for learning inverse dynamics, In 2010
IEEE international conference robotics and automation, pp. 2677-2682. IEEE, 2010.
Arya A Pourzanjani, Richard M Jiang, and Linda R Petzold. Improving the identifiability of neural networks for bayesian inference. In NIPS Workshop on Bayesian Deep Learning, volume 4, pp- 31, 2017.
Aniruddh Raghu, Maithra Raghu, Samy Bengio, and Oriol Vinyals. Rapid Learning or Fea- ture Reuse? Towards Understanding the Effectiveness of MAML. In International Confer- ence on Learning Representations, 2020. URL IkcMkCEtPB
net/ www.:nnttffffrmmmt... forum?io
Maziar Raissi, Paris Perdikaris, and George Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.
James B Rawlings. Tutorial overview of model predictive control. IEEE control systems magazine, 20(3):38-555 2000.
Sebastian Ruder. An overview of multi-task learning in deep neural networks. arXiv preprint 170.059706. 2017.
Shagun Sodhani, Amy Zhang, and Joelle Pineau. Multi-task reinforcement learning with context- based representations. In International Conference on Machine Learning, pp. 0779797779..MM 2021.
Mark W Spong, Seth Hutchinson, and Mathukumalli Vidyasagar. Robot modeling and control. John Wiley Sons, 2020.
Russ Tedrake. Underactuated Robotics. 2022. URL. http:://underacctuatedd..ss Imit.edu
Nelson Vithayathil Varghese and Qusay H. Mahmoud. A survey of multi-task deep reinforcement
learning. Electronics, 9(9), 2020. ISSN 2079-9292. doi: 10.3390cllectronies9991..666 URL
https: http:://wwww...
www wttp:wwwpppp.... com 2079 79 00110099992////1166363
Haoxiang Wang, Han Zhao, and Bo Li. Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation. International conference on machine learning, pp- 10991-11000 1002. PMLR, 2021.
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu. Generalizing to unseen domains: A survey on domain generalization. IEEE Transactions on Knowledge and Data Engineering, 2022a.
Rui Wang, Robin Walters, and Rose Yu. Meta-learning dynamics forecasting using task inference. Advances in Neural Information Processing Systems, 35:21640-21653, 2022b.

---

Published as a conference paper at ICLR 2024
Yiping Wang, Yifang Chen, Kevin Jamieson, and Simon Shaolei Du. Improved active multi-task representation learning via lasso. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. -35478. PMLR, 23-29 Jul 2023. URL https: http:s::/pprceeedin mll...mmr. V2C2/ mBG2//ang23b.ttm.
ml p.ess
Christopher Williams, Stefan Klanke, Sethu Vijayakumar, and Kian Chai. Multi-task gaus- sian process learning of robot inverse dynamics. D. Koller, D. Schuurmans, Bengio, and I.. Bottou (eds.). Advances in Neural Information Processing Systems, volume 21. Cur- ran Associates, Inc., 2008, URU/ttps::/prroceednss..ccm newriig...euuiispp wwt.eeurr.s...pppeereeeeeeeeeeee files/
Yuan Yin, Ibrahim Ayed, Emmanuel de Bézenac, Nicolas Baskiotis, and Patrick Gallinari. Leads: Learning dynamical systems that generalize across environments. Advances in Neural Informa- tion Processing Systems, 34:7561-7573, 2021.
Mohammad Asif Zaman. Numerical solution of the poisson equation using finite difference matrix operators. Electronics, 1 1(1)), 2022. ISSN 2079-9292. doi: 10.3390/electroniess11 11526iss11111335 152365. URL.
https://wwwt mdpi. com 2073-9292/11/15/2555
Luisa Zintgraf. Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning In International Conference Machine Learning, pp. 7693-7702.
PMLR, R, 2019.
12

---

Published as a conference paper at ICLR 2024
A PROOFS
Lemma 1. Let .1,..., UN, and w WT E R and let r' I r and r/,..... ,'V and wi..... ..... wp R' be two sets of vector of full rank, satisfying Vi,t, wi Tu w U/z Then there exist P. P.QE Re' xr such that wi = Pw; and of = Qu, Furthermore, QP Ip
Proof of Lemma Denoting by V E RN V E RR W E RRTr and W E RTxx the matrix representations of the vectors, the scalar equalities Vi, t, wy T V/ w', of take the matrix form
VW = =V'WT.
(A.1)
Since V' is of full rank, the matrix V (V'VTT) )- xx is well defined and left inverse of V Multiplying ( by V yields
W' = WP with p= V E x
(A.2)
Similarly,
V' = VQT with Q: W E R
(A.3)
Now compute QP WTT W'
Proof of Proposition yields the stated result.
7 Applying Lemma I of = v(x(0), Us o(z(1)), and we We, w/
P
The case where c, + 0 can be handled as follows. We augment &p and v, and and with an additional dimension, with the last components of o and w equal to and the last components of y and equal to and respectively. The augmented vectors satisfy the assumptions of Proposition [ provided the augmented vf and w/ span IR*+1_ The proposition then applies, and implies that the physical parameters ş be recovered with affine transform. This case is tackled experimen- tally in the capacitor experiment (Section 5.3). where x o 0 a fortiori since the electrostatic field is linearized around a nonzero value. The physical parameters are identified using an affine regression.
B EXPERIMENTAL DETAILS
B. ARCHITECTURES
All neural networks are trained with the ADAM optimizer Kingma & Ba (2015). For CoDA, we set dg r. chosen according the system learned. For all the baselines, the adaptation minimiza- tion problem (2.5) is optimized with at least 10 gradient steps, until convergence. For training, the number of inner gradient steps of MAML and ANIL is chosen to be 1, to reduce the computational time. We have also experimented with larger numbers of inner gradient steps. This improved the stability of training, but the cost of greater training time.
B.2 SYSTEMS
We provide further details about the physical systems on which the experiments of Section are performed.
B.2.1 POINT CHARGES
The charges placed at fixed locations in the plane at fixed location. The training inputs located in Ω = 1- ,, 1] x [0. 1] which discretized into 20 x 20 grid and the ground truth potential field is computed using Coulomb's law.
The training data is generated by changing each charge's value in {l...., 5]", hence T = 5 We have experimented on different settings with various numbers of charges, and various locations. In Section [.1] dipolar configuration is investigated, where = 3, and one of the charges is far away on the left and two other charges of opposite sign are located near x2 = 0. Gaussian noise of size 0.1 is added to the field values revealed to the learner in the test dataset.
The system is learned with a neural network of 4 hidden layers of width 16, with the last layer of size r For evaluation, the test data is generated with random charges drawn from a uniform distribution in [1,...,5]" and the data points drawn uniformly in Ω
13

---

Published as a conference paper at ICLR 2024
B.2.2 CAPACITOR
The space is discretized into a 200 300 grid. The training environments are generated with 10 values of the physical context o (a, n) [0, 0.5] [-0.5, 0.5] containing the angular and the positional perturbation of the second plate, drawn uniformly. The ground truth electrostatic field is computed with the Poisson equation solver of Zaman 2022) For evaluation, 5 new environments are drawn with the same distribution.
The system is learned with neural network of 4 hidden layers of width 64, with the last layer of size r = 72 + = 3.
B.2.3
CARTPOLE AND ARM
We have implemented the manipulator equations for the cartpole and the arm (or acrobot), follow- ing Tedrake (2022), and have added friction. The training data is generated by actuating the robots with sinusoidal inputs, with for each environment 8 trajectories of 200 points and random initial con- ditions and periods. At test time, the trajectories generated with sinusoidal inputs for evalutation, and with swing-up inputs for trajectory tracking.
Cartpole The pole's length is set to the varying physical parameters the masses of the cart and of the pole: & E {1, 2} x {0.2, 0.5). o T 4. For evalutation, the masses are drawn uniformly around (2, 0.3), with amplitude of 01,0. The system is learned with neural network of hidden layers of width 16, with the last layer of size r n
Arm The arm's length are set to 1, the varying physical parameters are the inertia and the mass of the second arm: Or f00.25.0.3 0.3, 0.4) x {0.9, 1.0, 1.3), so T = 9. For evalutation, the inertial parameters are drawn uniformly around (0.5, 1), with amplitude of 0.3). The system is learned with neural network of hidden layers of width 64, with the last layer of size = n = 4.
B.2.4 UPKIE
Information about the open-source robot Upkie can be found at htttps://itthub com/ basts-robott s wwccbots/upkk. /upkiel
We trained the meta-learning algorithm on balancing trajectories of 1000 observations, with 10 different values for Upkie's 's torso, ranging from 0.5 to 10 kilograms. For evaluation, the mass is sampled in the same interval.
The system is learned with neural network of 4 hidden layers of width 64, with the last layer of size r = n + 2 2
B.3 INVERSE DYNAMICS CONTROL
The Euler-Lagrange formulation for the rigid body dynamics has the form
M(4 + C(z. 4)4 + g(4) = Bu,
(B.1)
where q is the generalized coordinate vector, M is the mass matrix, C' is the Coriolis force ma- trix, g(g) is the gravity vector and the matrix B maps the input u into generalized forces (Tedrake 2022).
Inverse dynamics control is nonlinear control technique that aims at computing the control inputs of a system given a target trajectory { Spong et al. (2020). Using model ID for the inverse dynamics equation (B.I), the feedforward predicted control signal â = 1D(g q, ā). These feedfor- ward control values can then be combined with low gain feedback controller to ensure stability, as
à + K(q 9) K'(g Q)-
(B.2)
For the cartpole, we used K = K' = 0.5. For the robot arm, we used K = K' =1
B.4 ADAPTIVE CONTROL
In a time-varying dynamics scenario, CAMEL can be used for adaptive control and system identi- fication. Given a target trajectory, the task-agnostic component v of the model predictions can be computed offline. In the control loop, the task-specific component w is updated with the online least squares formula. The control loop summarized in Algorithm 2 where have assumed for simplicity. The estimated inertial parameters are deduced from the task-specific weights with the identification matrix 4.4
14

---

Published as a conference paper at ICLR 2024
Algorithm 2 Adaptive trajectory tracking
input trained feature map v(z), target trajectory s + a Offline control
for timestep 0 A 8
do
= compute E, (9.. 9:: 9.) compute features 8, == (T.)) end for
Control loop
Initialize Mo Ly. W) (0, (0....,0) 0)
for time step 1 S < H do
compute ú, Wg Us
compute = qs
play U, ú, Ke,
observe 1s+1. 1s+1
compute Vg == v(r,)) )
update M,+1 M,
MM,.(M,o.)
14 M.0
update Ws+1 Ws (Vs Ws Ma)
end for
15

---

Published as a conference paper at ICLR 2024
B.5 ADDITIONAL NUMERICAL RESULTS
We provide details concerning Table 2
Computational time For the computational times of Table 2 we arbitrarily chose the shortest time the time unit, for clearer comparison among the baselines. The computational times were measured and averaged over each experiment, with equal numbers of batch sizes and gradient steps across the different architectures. For training, the time was divided by the number of gradient steps.
Table 3: Adaptation performances with standard deviations.
Charges, 30 trials
Capacitor, 5 trials
System
3-shot
10-shot
5-shot
40-shot
MAML
4.1E-0 + 2E-0
1.6E- + 5E-2
N/A
N/A
ANIL
3.5E0 + 5E-1
9.2E-4 + 5E-4
4.4B-2 + 2E-2
3.6B-21 1B-2
CoDA
1.0E- + 9F-2
8.2R-2 + 3E-2
4.7B-2 + 5B-5
2.6B-2+ + 1B-2
CAMEL
2.0E-4 = IE-4
1.0E-4 + 5E-5
3.6B-2 2E-2
2.6B-2 + 1E-2
C-CAEE
3.0B-3
6.5B-2
E e-Capacitor,
= 0.1, 5 trials 30-shot
System
3-shot
MAML
N/A
N/A
ANIL
1.1B-3 + 5E-5
1.1E-3 = 5E-5
CoDA
1.2E-3 + 5E-4
1.0E-3 + 5B-4
CAMEL
4.2F-4 + 1R-4
1.9F-4 2R-5
P-CAMEL
1.9E-4
Cartpole, 50 trials
Arm, 50 trials
System
50-shot
100-shot
50-shot
100-shot
MAML
4.3E0 + 7F-1
3.5E0 + 6E-1
1.0E0 1E-1
8.1E-1
+ SE-2
ANIL
3.8E-1 - IB-1
2.5E-2 + 9E-2
8.5E- + 1E-1
7.5E- + 4E-2
CoDA
3.8E-1 + 9B-3
8. + 16-1
9.5B- + 96-2
9.3B-
+ 6E-2
CAMEL
4.8E-2 IB-2
3.1E-3 5E-4
3.1E-1 = 5E-2
2.4E- 1 = 1E-2
System
Upkie, 15 trials
MAML
1.5 7E-3
ANIL
1.9E-2 + 6E-3
CoDA
2.16-2 + 3E-3
CAMEL
8.2R-3 5R-3
16

---

Published as a conference paper at ICLR 2024
target
ANIL
MAML
CAMEL
CAMEL
Figure 6: 5-shot adaptation for the point charge system. Top. The four charges are positive, as in the training meta-dataset. Bottom Two of the four charges negative.
target
ANIL
CoDA
CAMEL
C-CAMEL
Figure 7: Capacitor, 40-shot adaptation.
CAMEL
MAML
200
250
.300
350
450
500
tine
Figure 8: Upkie torque prediction, 100-shot adaptation.
B.6 ZERR-SHOT ADAPTATION AND SCIENTIFIC DISCOVERY
Zero-shot adaptation Looking the problem from another angle, Proposition I also shows that w can be estimated linearly as a function of y,, at least when r = n (which ensures that is nonsin- gular). Computing estimator of function of &o with the inverse regression (.4) enables
17

---

Published as a conference paper at ICLR 2024
zero-shot (or physical parameter-induced) adaptation scenario: when an estimate of the physical parameters of the new environment known priori, a value for the model weights can be inferred. We call this adaptation method c-CAMEL.
In a data-driven approach, training CAMEL offers not only the ability to adapt to a small number of observations, but also to predict the system without any data for arbitrary values of the its parameters. We believe that the O-shot adaptation algorithm y-CAMEL can be used in the process of scientific discovery. In many cases, the experimenter has the knowledge of (or knows an estimate of) the physical quantities varying across experimental conditions, while not knowing accurately the system itself. Then, g-CAMEL can be used to infer the target function for chosen values of the physical parameters independently of the values observed for training.
Of course, the predictions of g--AMEL are good only if the estimator & (4.4) is good, implying sufficient number of training tasks and an effective training of CAMEL. For nonlinear physical contexts, the values of yp that are investigated should be close to the reference value yo so that (422) holds.
We further illustrate on the toy example of n = 4 point charges, for which the experimenter could observe experiments with positive charges. Figure Figure[Jshows shows the predictions after 5-shot adaptation of the different meta-models, along with the zero-shot adaptation of s--CAMEL We can see that only CAMEL and ş-CAMEL adapt well negative charges. In particular, the zero-shot adaptation of P-CAMEL enables estimating the system in an experiment whose numerical values are completely different from the training dataset, thanks this case (since they are known to be linear in the charges). Importantly, evaluating c-CAMEL for different values of oo is not costly, since the identification map is already computed using the training data.
the structure of the model and of the equations in
We could imagine that this scenario might enable discovering new properties complex physical systems as by exploring the space of physical parameters, in data-driven fashion. Regarding the simple example of Figure 6 knowing the form the electrostatic field in this quadrupole setting underlies the understanding of Penning's 's ion trap Kretzschmar [1911)
18


Reviews:
[{"review_id": "Skunu13TWU", "reviewer": "Reviewer_rUF2", "summary": "This article presents a new meta-learning method, CAMEL, which uses affine task-specific parameters. It is motivated by the application of modelling physical systems, many of which are described in the article. CAMEL is demonstrated on these modelling examples and a robotic control task, with comparisons to state-of-the-art meta-learning methods MAML, ANIL, and CoDA. The interpretability of the method as a means of discover physical parameters is discussed.", "questions": "How many trials were used to compute the averages in Table 2?\n\nWhat do the \"Training\" and \"Adaptation\" columns correspond to in Table 2?\n\nHow many gradient steps were used in the various benchmarks?\n\nWere the hyperparameters of the Adam optimizer the same for all benchmarks?\n\nCould common physical parameters be learned in $v_*(x)$? What guarantees that $v_*(x)$ is task-agnostic?\n\nDo the authors consider that their experimental results validate the hypothesis of equation 3.1?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The article is well-motivated and well-written. Both the motivation towards interpretable data-driven modelling of physical systems, and the motivation of simplifying meta-learning approaches like MAML, are well justified.\n\nThe proposed CAMEL method appears to be a simple but effective method for meta-learning. The overview of meta-learning approaches, comparing MAML, CoDA, and CAMEL, in section 2.2 provides a useful overview of the different methods. The use of different $w_t$ vectors for different environments is motivated in the text and suitable for physical applications, especially as $w_t$ can be learned used least squares.\n\nThe experimental evaluation of CAMEL is convincing, with a number of different physical modelling systems used for demonstration. CAMEL appears to match or outperform other meta-learning approaches on all tasks, although further details could improve the experimental evaluation (as detailed below). The code is provided and is simple to understand, using PyTorch for single-file implementations of CoDA, MAML/ANIL, and CAMEL.", "weaknesses": "The experimental results rely heavily on Table 2, however the experimental conditions for these results are not made clear. Specifically, the training details for MAML, ANIL, CoDA, and CAMEL are not fully detailed. The \"Training\" and \"Adaptation\" columns are not described, nor are their units present. The caption states that the table presents the \"Average adaptation mean squared error and computational time,\" but the number of trials over which an average is taken, the standard deviation, the units of computational time, and how computational time was measured are not given. Statistical tests on the significance of values presented in Table 2 would be helpful.\n\nThe argument that CAMEL is an interpretable method relies on the idea that the weight vector $w_t$ learned on the training meta-dataset can correspond to physical parameters of a system. This is explained in Proposition 1, which assumes that the training loss is taken to 0. The authors do point out that identifying the meaning of each parameter, in other words, interpreting the parameters, is out of reach for black-box meta-learning architectures. However, the argument that CAMEL allows for such interpretation is not sufficiently demonstrated in my opinion. The example of learning an electric point charge system is given in section 5.1 and expanded in section B.5, however it relies heavily on the fact that CAMEL is able to learn the dynamics of the system, and not on the interpretability of the resultant parameters. Demonstrating how the learned physical parameters allow for an interpretation of the model and physical system, beyond the performance of CAMEL, would help make this argument. Physics informed neural networks (PINNs) share the same motivation and have been used to present interpretable results; their inclusion in this section and discussion would further improve the interpretability argument. \n\nLastly, the authors appear to only study the case in which $c = c_* = 0$. This is a large simplification. If Equation 3.1 is valid, I believe this means that $h(x;\\theta_0) = 0$, which is quite a departure from other meta-learning methods. If this is the case, the authors should further justify it and potentially explore the case where $c \\neq 0$. If it is not the case, then clarification is needed in section 4.3 as to when this simplification is made.\n\nWhile the article is well-written, it suffers from two minor presentation weaknesses. The first is a reliance on popularity as justification; the popularity of a method does not justify its use. The sentence \"Given the popularity and expressiveness of neural networks, incorporating multi-task learning into their gradient descent training algorithms is a major challenge\" (2.2) is an example. The challenge of multi-task learning is not dependent on the popularity of neural networks, so why is it used as an opening clause? The second formatting issue is minor: works are cited in text when they should be cited in parentheses (although there are also proper uses of in-text citations). See, for example, this paragraph in section 4, where the Ljung citation is correct but the Nelles citation isn't:\n\"System identification and model identifiability are key issues when learning a system (Ljung, 1998). Although deep neural networks are becoming increasingly popular for modeling physical systems, their complex structure makes them impractical for parameter identification in general Nelles (2001).\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ocFyfzbbiU", "reviewer": "Reviewer_3DG3", "summary": "The paper investigates meta-learning physical systems via\nassociating every task with a linear embedding/task weight\nw that impacts the predictions: this is shown in equation 3.2\nwhere the prediction involves a global prediction\nfollowed by a modification from the linear part.\nAt test-time, the task weight of a new task is found\nby gradient descent on the task loss.\nSection 4 goes on to describe many settings where it is\nreasonable to have this linear part of the model,\nand prop 1 shows that the task weights can recover\nthe true dynamics of the system, which leads to the\nan interpretability of what the task weights are.\nThe experimental result show a number of settings\nfrom physics and reinforcement learning where\nthe method is able to learn better meta-dynamics models\nthan MAML, ANIL, and CoDA.", "questions": "Overall I think this is a well-executed paper with some\ngood modeling ideas and experimental insights.\nI gave the paper a weak reject due to the experimental settings\nnot being directly comparable to the related work and the\nbaselines not being tuned as much as they could have been,\nand I am especially open to re-evaluating this assessment\nafter hearing the author's perspective on this.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "1. This is a reasonable model when there is some reason to believe\n   the true model follows the formulation of equation 3.2.\n   It could be interesting to further speculate about the\n   representation capacity or universality of this model.\n   (Section 4 is perhaps a good start to this)\n2. The experimental results clearly demonstrate the\n   capability of the model, especially in how well CAMEL\n   is able to attain a low identification error in Fig 2.\n3. CoDA is a great baseline method to compare against,\n   in addition to MAML/ANIL, and Table 1 clearly summarizes\n   the modeling differences.", "weaknesses": "1. Every experimental meta-learning setting in this paper\n   seems to be newt\n   This would have been helpful to help better-ground the results\n   in the community, as otherwise it is difficult\n   to understand if the performance improvements are coming\n   from improper implementations of other methods\n   or from a fundamental modeling advancement.\n   For example, MAML with a training step size of 30 and\n   adaptation size of 10 in Table 2 is very arbitrary and\n   unlikely to be the best MAML hyper-parameters here.\n\n   The paper would be significantly more convincing\n   if the CAMEL model was evaluated in the exact experimental settings\n   from CoDA or [Wang et al., NeurIPS 2022](https://arxiv.org/abs/2102.10271),\n   which also meta-learn physical systems with varying amounts of\n   contextual information.\n2. There is a significant amount of omitted related work on\n   meta-reinforcement learning that is also adapting controlled\n   MDP dynamics and policies to multi-task RL settings,\n   e.g., [Nagabandi et al., ICLR 2019](https://arxiv.org/abs/1803.11347)\n   and the citation graph around it.\n3. The term \"interpretable meta-learning\" in the title and\n   throughout the rest of the paper is a big statement without\n   the right qualifications. The paper gets to defining what\n   interpretability means here in proposition 1 of recovering\n   true parameters when they exist and fit into the modeling\n   structure they have set up, but I this is not what I had\n   in mind when I first saw the term \"interpretability\".\n   It also seems like there is still a large amount of\n   uninterpretability in the system, as the model in\n   equation 3.2 still has 2 neural networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dPVoLkgxCr", "reviewer": "Reviewer_X11Z", "summary": "The paper proposes an approach CAMEL for the adaptation to new tasks by first learning a shared neural network feature extractor together with separate linear heads for each training task, then at test time perform least square regression to compute the linear head weights given a new task’s labelled data. The authors show this formulation can allow physical parameters identification for linearly parameterized systems under certain conditions. Experimentally, the authors show that their approach can outperform meta-learning baseline methods (MAML, ANIL, CODA) in terms of generalization, computational speed, and interpretability.", "questions": "- How many steps of gradient adaption is used for ANIL? It is important to note that one can apply more steps of adaptation during evaluation than training as it can potentially improve ANIL's performance.\n- Can the authors explain for example 1 and 4, what is the learning goal here? I saw the authors mention $x=(q, \\dot q, \\ddot q)$ and $y=u$, but can the authors explain why do we want to predict $u$ (which I'm assuming is a function of time) in the first place?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The interpretability and physical parameter identification aspects this paper focuses on are important and often overlooked by existing meta-learning methods.", "weaknesses": "- **The proposed method lacks novelty and are not situated in the right context**. The authors claim to propose a novel “model agnostic meta-learning architecture.\" However, the proposed approach is not model agnostic (because it requires specifying the width of the last layer dimension) and more importantly is not novel and not situated in the most relevant context.\n    - **Novelty**: It could be argued that the proposed method is not meta-learning but applying multi-task representation learning in a learning-to-learn problem. A more general version of the authors’ proposed approach have been theoretically studied in terms of its generalization performance by Maurer et al  in 2016 (page 7, section 2.2 Bounding the Excess Risk for Learning-to-learn). More recently, Wang et al 2021 have proposed almost exactly the same approach by first multi-task learning a shared feature extractor with linear heads and then performing last linear layer fine-tuning over a new task’s data at test time (section 3.4 Fine-tuning for Test Task Adaption). Besides, Wang et al has also studied the computational time savings of this MTL approach compared to last-layer MTL methods. Thus it is not correct for the authors to claim their methods as a novel approach.\n    - **Context**: In the background section, the authors situate their methods among Gradient-based Meta-learning methods. However as the authors have shown in Table 1, CAMEL doesn’t perform gradient adaptation, so it’s not a gradient-based method. Instead, because CAMEL adapts only the last layer of the learned model, a more appropriate way to situate the current method is to compare it among last-layer meta-learning methods, which includes not only ANIL (which the authors have compared against) but more importantly MetaOptNet (Lee et al 2019) and R2D2 (Bertinetto et al., 2019). Here both MetaOptNet and R2D2 solve the last linear layer exactly (using a convex solver or closed-form expression) and use implicit function theorem to propagate the gradients to the earlier feature extractors. Because this paper focuses on the regression task, the authors should compare additionally against R2D2 in the experiments. (One can think of R2D2 roughly as ANIL where the last layer adaptation are solved exactly instead of using a fixed number of gradient descent steps.)\n    \n    Maurer, A., Pontil, M., and Romera-Paredes, B. The beneﬁt of multitask representation learning. The Journal of Machine Learning Research, 2016.\n    \n    Wang, H., Zhao, H., & Li, B. Bridging multi-task learning and meta-learning: Towards efficient training and effective adaptation. In International conference on machine learning, 2021\n    \n    Bertinetto, L., Henriques, J. F., Torr, P., and Vedaldi, A. Meta-learning with differentiable closed-form solvers. In International Conference on Learning Representations, 2019.\n    \n    Lee, K., Maji, S., Ravichandran, A., and Soatto, S. Metalearning with differentiable convex optimization. In CVPR, 2019\n    \n- **Notation needs to be more consistent and clearer.** For example, both $f_t(x; \\pi)$ (page 3) and $f_t(x)$ (page 4) are used. Besides, the ground truth function is denoted $v_{\\star}$. However, it has $n$ output dimensions, while the function $v$ has $r$ output dimensions.\n\n- **Proposition 1** are stated without all the assumptions of the relationships between $r$, $n$, $T$, and $N$ (some of these relationships are scattered in earlier parts of the paper), making the statement difficult to parse. Besides, the assumption that $c = c_{\\star} = 0$ to me seems a bit restrictive as the ground truth $c_{\\star}$ are inherent to the physical process and could be nonzero. However, I’m not sure if the authors can make the same type of guarantees for the cases when these values are non-zero (and possibly unequal).\n- In the computational benefit section, the authors mention “adaptation at test time … is guaranteed to converge if the number of samples is greater than $r$.” This statement is a bit confusing, as the optimization is an ordinary least square which can be solved analytically without a notion of convergence. Maybe the authors mean to say when the number of samples is greater than $r$, the ordinary least squares is likely to be fully specified, thus admitting one unique solution. If this is the case, the authors should also discuss how to pick the solution when the ordinary least squares is underspecified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)