Leveraging Explanation to Improve Generalization of Meta Reinforcement Learning
Authors: Shicheng Liu
Minghui Zhu
Venue: ICLR 2026 Poster (2026)
Keywords: meta-reinforcement learning
generalization
theory
Primary area: reinforcement learning

Abstract: A common and effective human strategy to improve a poor outcome is to first identify prior experiences most relevant to the outcome and then focus on learning from those experiences. This paper investigates whether this human strategy can improve generalization of meta-reinforcement learning (MRL). MRL learns a meta-prior from a set of training tasks such that the meta-prior can adapt to new tasks in a distribution. However, the meta-prior usually has imbalanced generalization, i.e., it adapts well to some tasks but adapts poorly to others. We propose a two-stage approach to improve generalization. The first stage identifies "critical" training tasks that are most relevant to achieve good performance on the poorly adapted tasks. The second stage improves generalization by encouraging the meta-prior to pay more attention to the critical tasks. We use conditional mutual information to mathematically formalize the notion of "paying more attention". We formulate a bilevel optimization problem to maximize the conditional mutual information by augmenting the critical tasks and propose an algorithm to solve the bilevel optimization problem. We theoretically guarantee that (1) the algorithm converges at the rate of $O(1/\sqrt{K})$ and (2) the generalization improves after the task augmentation. We use two real-world experiments, two MuJoCo experiments, and a Meta-World experiment to validate the algorithm.

Published as a conference paper at ICLR 2026
LEVERAGING EXPLANATION TO IMPROVE GENERAL- IZATION OF META REINFORCEMENT LEARNING
Shicheng Liu & Minghui Zhu Department of Electrical Engineering Pennsylvania State University University Park, PA 16802, USA (sf15539, muzl 6) @ppusuu.. edu
ABSTRACT
A common and effective human strategy to improve a poor outcome is to first identify prior experiences most relevant to the outcome and then focus on learn- ing from those experiences. This paper investigates whether this human strategy can improve generalization of reetrrreifooremment learning (MRL). MRL learns a meta-prior from set of training tasks such that the meta-prior can adapt to new tasks distribution. However, the meta-prior usually has imbalanced general- ization, i.e., it adapts well to some tasks but adapts poorly to others. We propose a two-stage approach to improve generalization. The first stage identifies "crit- ical" training tasks that most relevant to achieve good performance on the poorly adapted tasks. The second stage improves generalization by encourag- ing the meta-prior to pay more attention the critical tasks. We conditional mutual information to mathematically formalize the notion of "paying more atten- tion". We formulate a bilevel optimization problem maximize the conditional mutual information by augmenting the critical tasks and propose an algorithm to solve the bilevel optimization problem. We theoretically guarantee that (I) the algorithm converges at the rate of O(1/VK) and (2) the generalization improves after the task augmentation. We use two real-world experiments, two MuloCo experiments, and Meta-World experiment to validate the algorithm.
INTRODUCTION
Meta-reinforeement learning (MRL) learns a meta-prior from a set of training tasks where each training task is an RL problem and is drawn from an implicit task distribution. The predominant approach in existing works (Beck et al., 2023) is to learn meta-policy as the meta-prior. In this paper, we follow this standard setting and denote the learned meta-policy by To- The goal is for To to generalize effectively across the task distribution. However, both prior works (Dhillon et al. 2019 Nguyen et al.) 2021 Yu et al.] 2020) and our empirical findings (see Appendix M.9) indicate that To usually adapts well to some tasks but poorly others. This paper proposes method to improve generalization of the meta-policy xn- Our method is inspired by an effective strategy that humans commonly use in daily life improve a poor outcome, where humans first identify prior experiences most relevant to the poor outcome and then focus on learning from these experiences to improve the outcome. For example, if a student fails some problems in an exam, common improvement strategy for the student is to find similar problems from previous homework and focus more on these problems in future study. Our approach consists of two stages. The first stage identifies "critical" training tasks most important to the poorly adapted tasks. The second stage encourages no to focus more on the critical tasks to improve generalization of πo. Note that our approach operates in post. hoc setting, i.c., after the MRL algorithm has already produced a meta-policy no.
The first stage proposes an example-based explanation method to identify the training tasks most relevant to the poorly adapted tasks. Example-based explanation is widely used in explainable ma- chine learning (Caruana et al. 1999 Sun et al. 2024) to explain a model's decision through relevant examples. It is inspired by the observation that humans usually relevant experiences interpret a new thing (Crabbé et al. 202I). In our case, we formulate a bilevel optimization problem where the upper level learns weight vector weight each training task such that the corresponding weighted

---

Published as a conference paper at ICLR 2026
meta-policy performs best the poorly adapted tasks and the lower level learns this weighted meta- policy. The training tasks with highest weights are the most important/relevat tasks to achieve good performance on the poorly adapted tasks. We refer to these training tasks "critical" tasks.
The second stage improves generalization by encouraging the meta-policy to pay more attention to the critical tasks. We mathematically formalize the notion of "attention" through information- theoretic lens. Specifically, we use the mutual information between the meta-policy and the critical tasks to quantify the task information of the critical tasks stored in the meta-policy (Yin al.. 2019 Yao et al., 202T). An increase in mutual information indicates that the meta-policy stores more information of the critical tasks and thus pays more attention to the critical tasks. To increase this mutual information, we propose to augment the critical tasks by generating augmented data. The augmented data enhances data diversity of the critical tasks and contains additional information. Therefore, it is expected that the meta-policy trained on the augmented data stores more information of the critical task. Data augmentation has been applied to RL ( (Wang et al., 2020 [Lskin et al. (020) and meta-learning (Yao et al. 2021: Rajendran et al. 2020), but these methods use predefined rules to augment the data. While the predefined augmentation rules can increase the task information of the critical tasks stored in the meta-policy, they do not maximally increase such stored information. Motivated by (Yin et al. 2019) which improves generalization by maximizing mutual information between the task data and meta-parameter, we formulate bilevel optimization problem. In the upper level, we learn how to augment the critical tasks to maximally increase the stored information. To achieve this, we use conditional mutual information (CMI) to quantify the additional information of the critical tasks stored in the meta-policy after the task augmentation, and learn an augmentation method to maximize CMI. The difficulty of the upper-level optimization of the meta- motayp Therefore, the lower level formulates a distributional optimization problem where a meta-polle meta-policy distribution corresponding to the current augmentation learned.
compute a distribution
We include related works in Appendix A and summarize our contributions as follows.
Contribution statement. This paper proposes leverage explanation to improve generalization of the specific meta-policy No. Our contributions are threefold:
First, we propose an example-base explanation method to identify the critical training tasks that are most important/relevant to the poorly adapted tasks as an explanation.
Second, we introduce an information -hformation-thhoreticc theoretic framework and formalize the problem of leveraging the explanation to improve generalization as a bilevel optimization problem. The upper level learns how to augment the critical tasks to maximize the conditional mutual information, and the lower level computes the meta- meta-policy distribution corresponding to the current augmentation. We propose an algorithm to solve the bilevel optimization problem.
Third, we theoretically guarantee that (i) our algorithm converges at the rate of Q(IVK)) and (ii) the generalization improves after the task augmentation. We use two real-world experiments, two MuJoCo experiments, and a Meta- World experiment to empirically validate that our algorithm can improve the generalization of the meta-policy To.
2 PRELIMINARIES
Reinforcement learning. An RL task T; is based on Markov decision process (MDP) M; (S, A. y; Pt, M;, r:) which includes a state set S, an action set A, discount factor y E (0,1), state transition function P((+l,,)). +), an initial state distribution v.(.), and a reward function r.))))) RL learns a policy Te to maximize the cumulative reward max, ET 7 dt]|so |80 ~ 1/)..
The policy gradient E( [V log T (als) (s, a)] where AT is the advantage function under the reward r; and policy т, p² (s,
E" 11ss = 8, az =aH|so ~ vi] is the
stationary state-action distribution of the policy π, and I(-) is the indicator function. Based on the policy gradient, we can formulate a surrogate objective for RL (Wang et al., 2020): : (((T) & E(s.a)~p= [log iT ((a)s (A)s)A]( a)]. For brevity, we omit the explicit notation of the policy parameter yp.
Meta-reiiforremmnt learning. MRL aims to efficiently solve multiple RL tasks by learning a meta-policy. The meta-policy is learned from a group of N training tasks N" sampled from an implicit task distribution P(T). It is typically assumed (Beck al. 2023 that different tasks
J==1
2

---

Published as a conference paper at ICLR 2026
share (,) but may have different (P. 2 r.).. Here, the superscript "tr" means that these com- ponents belong to training tasks. Later on, we will use different superscripts to represent different kinds of tasks. Current mainstream MRL works (Beck et al., 2023 Finn et al. 2017 Fallah et al. 2021
Xu et al. 2018 Liu et al. 2019) have the following bilevel structure:
1 N
max L(O, (T
(T))(
(T) Alg(To, T,"):
where the upper level learns meta-policy we such that the corresponding tusk-specific adaptation (0) can maximize the cumulative reward (x]()(() on each training task 7,,", and the lower level computes the task-specific adaptation (0) induced from no. Different meta-learning methods use different algorithms to compute the task-specific adaptation =²)( Here, we use Alg (T)) to generally represent an algorithm that computes the task-specific adaptation.
We denote the meta-policy learned from T by no. and evaluate its generalization by sampling set of new tasks from P(T). For each sampled task, we perform task-specific adaptation and measure performance using the resulting cumulative reward. However, as noted in the prior work (Yu et al.) 2020) and confirmed by our experiment (see Appendix [M9),, only subset of the adapted policies achieve high cumulative reward, while others perform poorly. Therefore, we can find N7porr poorly adapted tasks and denote them by the set (7,ptM adapted tasks in Appendix M.
We include the details of how to find poorly pour "
EXAMPLE-BASED EXPLANATION
This section proposes an example-base based explanation method. The proposed method is motivated by the recent advances in example-basee explanation for RL. For example, (Liu & Zhu (025) identifies the state-action pairs that are most important to suboptimal performance an explanation and (Liu et al., (225b) identifies the preference data that is most important to unsatisfactory responses as an explanation. Inspired by these approaches, we extend the idea to the MRL setting and aim to identify the training tasks that are most important for the meta-policy to achieve high cumulative reward on the poorly adapted tasks [7,p0orr J. N Ji=1 (after adaptation). We refer to these training tasks as "critical tasks" and aim to identify the top N critical training tasks as an explanation. For this purpose, we propose to learn an importance vector E R where each dimension captures the importance of the corresponding training task T," for improving cumulative reward on (T, poor N The problem is formulated as the following bilevel optimization problem:
li=1 1
N
max L(0 (w), (T, poory N
J )
s.t. 8" (w) = arg max
(
(2)
where the upper level learns how to weight each training task such that the corresponding weighted meta-policy (w) can adapt to (T, j with maximum cumulative reward, and the lower level computes the weighted meta-policy Tg. (w)(w) corresponding the current weight w. We include the algorithm to solve the problem (2) in Appendix B The algorithm is single-loop where at each it- eration, start from the parameters in the previous iteration and only partially solve both the upper-level and lower-level problems via one-step gradient ascent.
We denote by w" an optimal solution of the problem (2). A larger value of w means that the weighted meta-policy (w') assigns more importance to the training task T.",, indicating the high No Accordingly, we define the top Neri training tasks with the highest weight values as the critical tasks, denoted by (T."") N
relevance of this task to achieve high cumulative reward on (T.puor) J==1
4 GENERALIZATION IMPROVEMENT VIA TASK AUGMENTATION
This section uses the explanation (i.e., the critical tasks {T.00 ( N in Section to improve general- ization by encouraging the meta-policy To to pay more attention to the critical tasks. One may be concerned that paying attention to the critical tasks can degrade performance on other tasks, we include evaluation in the experiment to demonstrate that method only degrades very few (less
3

---

Published as a conference paper at ICLR 2026
than 5%) tasks but the average performance on all the tasks always improves. The key challenge to encourage the meta-policy to pay more attention to the critical tasks lies in how to mathematically formalize the notion of "paying more attention"..
A natural approach is to assign higher weights to the critical tasks. However, existing task weighting methods (Yao et al., 2021 Cai et al.) 2020) typically require additional target task set to guide how to assign the weights. As a result, these methods aim to generalize specifically to the given target task set. In contrast, we do not have a target task and our goal is not to generalize well to a specific target task set. While solving the problem ( yields a weighted meta-policy (T-(w-) that improves generalization on (T: f= N the generalization over the task distribution may not improve. In Section 5
we empirically demonstrate that our method outperforms the task weighting method.
We study the notion of "attention" from an information theooretc perspective. If the task information of the critical tasks stored in the meta-policy increases, it means that the meta-policy pays more attention to the critical tasks (Rajendran et al. 2020). To achieve this, we propose to augment the critical tasks by generating augmented data. The augmented data contains additional information and diversifies the original training data, therefore, training on the augmented critical tasks can store more information of the critical tasks in the meta-policy (formally proved in Theorem 2
Inspired by the empirical success of mixup augmentation in improving generalization in supervised learning (Zhagg et al., 2018), meta-learning (Yao et al. 2021]. and RL (Wang et al. 2020), we
adopt mixup to augment the critical tasks. Recall from Section 2 that the surrogate RL objective (a|s)) A7 (8,a), a)}. where p" (s, a) is the stationary
for critical task 7, is: (TT) = ((s.a)>pp [log <
state-action distribution of the policy x. We define the stationary state distribution of x as (s) &
IaEA (s, da. To train a policy, we collect transition tuples (s, T, Snext) where ~ p= (-), the policy π selects action a, and the environment returns reward r and next state Spea- Given two sampled states 8' ~ p-).), mixup generates an augmented state 7;S + (1 - >s)s'' where the mixing coefficient X E [0, of the critical task T, is sampled from distribution P()). In this paper, we use A, to denote random variable that follows the distribution P()) and use X to denote specific value sampled from P(A). Note that our augmentation method assumes that the augmented state is always feasible. In practice, the augmented state is always feasible in experiments (explained in Appendix [M.).. The policy æ then selects an action ā at the augmented state s, and executes this action to collect the augmented tuple (3, a, F, 8nex). Since the augmented state 3 is different from the original two states 8 the augmented tuple (3, ā, F, 7,8") contains different information from the original two tuples ((8, a, T, (oex) and (s', a', r,,,, ( Therefore, adding augmented tuples will diversify the original training tuples and contain additional information.
For specific A,, the mixup augmentation induces an augmented stationary state-action distribution gT.A (..)) whose expression is in Appendix C This gives rise to an augmented task 7, (Aa),, with the augmented surrogate objective defined as Jeth Ar) _ E(s,a))-pm As [log 7 ((]))) AT (8, a)]- Given
the augmented critical tasks, the meta-objective (i.e., the upper-level objective) in T becomes:
N
N N
L(0, (T:" (Aa)) (T") N
Jed ( (0), A
Σ ((([()) (3)
N
In contrast to the original meta-objective in I the new objective (3) replaces the original critical w tasks {7,-tt Ji=1 with the augmented critical tasks (Ac) Since A; is a random variable drawn from P(A), the corresponding augmented task (A,) is also a random variable. In the following context, we use the notation T, ~ P())) to highlight this stochasticity.
Remark (Augmentation is different from simply sampling more data). Augmentation is dif- ferent from using the policy n to sample more data. The reason is that the additional data sampled by will always follow the original state-action distribution p² and thus the RL objective remains as the original objective Jen (T). ). In contrast, augmentation changes the state-action distribution to pr and thus leads to a different optimization objective J.ri X4).
In this section, we study the problem of "paying more attention" from the perspective of "storing more information". We have intuitively explained that the task augmentation is expected to store more information of the critical tasks in the meta-policy because the augmented data diversifies the original training data and contains additional information. In the following context, we formalize this intuition and mathematically study the problem from an information- perspective. This
4

---

Published as a conference paper at ICLR 2026
section has two parts. The first part formulates bilevel optimization problem to learn how to augment the critical tasks to maximally increase the task information of the critical tasks stored in the meta-policy. The second part proposes algorithm to solve the bilevel optimization problem and theoretically proves that the generalization improves after the task augmentation.
4.1 PROBLEM FORMULATION
In this part, we formulate bilevel optimization problem that explicitly captures the goal of "pay- ing more attention" to the critical tasks by storing more task information in the meta-policy. As previously discussed, augmenting the critical tasks is expected to increase the amount of task in- formation stored in the meta-policy. We now formalize this intuition by introducing the following enformation-theoreti definition:
Definition 1. The additional information stored in the meta-policy Bg after the task augmentation can be quantified by [(;; (A; ~ P( (T,a)E Nei) which is the conditional mutual in- formation between the meta mneaa-parameter and the augmented critical tasks (T, P())) N given the original critical tasks (T N
In information theory (Wyner 1978). the conditional mutual information quantifies the difference between the information that 0 and {7, ~ ) share and the information that 0 and (T700) Ti=1 share. In other words, it quantifies the amount of additional information stored in A by additionally knowing (T: (A, P(
Ner
Ji=1 is already known. Therefore, [0; (A; ~ P(A)) No (T. )EE) > 0 means that the information of the critical tasks stored
Neri given that [T
N in the meta-parameter increases after we augment (T, Ji=1 (T," P(A)))
Our objective to augment the critical tasks to store more information, ensuring that I(0: (A, P(A)) NoH (7,"") (T))) > O. While the current mixup methods Yaao et al. 2021 Wang et al., (020)) use predetermined distribution P()) of X; to mix the data, these methods do not guarantee that the resulting augmentation maximally increases the stored task information. Mo- tivated by (Yin et al., 2019) that improves generalization by maximizing the mutual information between the task data and meta-parameter, mizing the distribution P(A) to maximize the conditional mutual information. To this end, we model the distribution of À parameterized distribution Pox (A) with parameter dx- Our objective is to optimize the distribution parameter ox to maximize the following conditional mutual information:
propose to learn an optimal augmentation by opti-
I(0: (A, Pox (A))
log ("OT log P (*1( N
where the derivation is in Appendix D Here, (AI), is the posterior distribution of
the meta-parameter given the augmented critical tasks (T. (Ai) 1==1= while P* ('((.((,"") ) is the posterior distribution of & if only the original critical tasks (T, Ji=l are known. The difference log PM (0, (A) log Pa (O (,""" measures how much observing the augmented crit- ical tasks changes our belief about the meta-parameter & compared to observing only the original tasks. If the augmented tasks are informative, the posterior Pa (0) concentrated, yielding larger log-likelihood for plausible values of 0: if the augmentation provides no additional information, the two pog-posteriors coincide and the difference becomes zero. For simplicity, we omit the dependence on the non-critical training tasks (i.e., other training tasks that are not critical tasks), as they remain unchanged during augmentation.
Nei
Apoi
(A.)) becomes more
To maximize the conditional mutual information 4 we need to compute the posterior distributions p (0) (T A and Pa (0, (A.)) Analogous YYnn al. 2019), we treat 0 as a random variable where the randomness comes from the training stochasticity, Mathematically, the posterior
distributions are (the derivation in Appendix
P (-1TT," (A.) Nol
arg max Epp(o) [(0 (A) Nd (""~~~
+
5

---

Published as a conference paper at ICLR 2026
(*11TT;"") = EN (
(' (A)) (A))))))
(5)
where Po(0) is distribution of 0 parameterized by σ. The posterior (-1 (=)) can be straightforwardly obtained by optimizing the meta-objective (3) because the meta-parameter ) is directly trained over the augmented tasks. However, the posterior P (*, v cannot be no longer trained on the vanilla training tasks (without augmentation). To obtain the posterior P (-117,,"" Ji=1), we need to marginalize over all the possible augmentations and thus it is expectation of (A.) over all possible mixing coefficient {A, NoT By combining (4 and (5. we reach the bi-level optimization problem:
directly computed because the meta-parameter
max (0; (T, Pax (A)) (T, M dx
s.t.
Problem
(5)
The bi-level optimization in (6) captures the interaction between choosing good augmentations and evaluating how those augmentations influence the meta- parameerr. The upper level optimizes the augmentation distribution Pox. (A) to find mixing coefficients that make the augmented critical tasks contain as much additional information about the meta-parameter possible. However, evaluating how informative an augmentation is requires knowing how the belief over changes after observing the augmented tasks. This is handled by the lower-level problem, which computes the posterior distributions over 0 under both the augmented critical tasks and the original tasks. essence, the lower level tells what the learner would believe for given augmentation, and the upper level uses this feedback to adjust the augmentation distribution in order to maximize information gain. Thus, the bi-level structure reflects that we must jointly learn (i) how mix the tasks and (ii) how these mixed tasks influence the meta peta-paramettr distribution.
4.2 ALGORITHM
In this section, we develop an algorithm to improve the generalization of MRL. The algorithm consists of two major stages: identifying critical tasks and solving the bi-level optimization problem in (6) to improve generalization. We elaborate the algorithm as follows:
Algorithm Explainable meta reinforcement learning to improve generalization (XMRL)
Input: Initial mixing coefficient distribution Pox,o (A) and meta-parameter distribution Poo (0),
training tasks (T. ( and poorly adapted tasks {T,peur N Ji=1
Output: Learned mixing coefficient distribution P (o and meta-parameter distribution
P (1Ac, 1, (0),
1: Generate the explanation (i.e., the critical tasks (T." using the algorithm in Appendix B
2: for = 0.... K 1 do
3: Lower-level optimization: Sample N sets of coefficients (AG, L.
ii= from Pax. (A); for each
set, compute the distribution parameter o" (1X 1N such that (0 (A ( =
( Nai (0). Estimate P (- J. No ΣxT P" (ITTT" M
4: Upper-level optimization: Compute the hyper-gradient I in Lemma | and update the mixing coefficient distribution parameter ΦA = ΦA.E + -.
5: end for
We begin by generating explanations for the original meta-policy mo and extracting the critical tasks (T, J y using the algorithm in Appendix B These tasks are the ones to which the meta-policy To should pay more attention, and therefore they serve as the targets for augmentation.
Stage Identify critical tasks (line | in Algorithm D
Stage 2: Solve the bi-level optimization problem (6) to augment eritical tasks. Once the critical tasks are identified, the algorithm solves the bi-level optimization problem (6) via lower-level and upper-level optimization. We elaborate the lower-level and upper-level optimization follows:

---

Published as a conference paper at ICLR 2026
Lower-level optimization (line 3) At iteration k the lower-level step is to compute the posterior distribution over 9 under both the augmented critical tasks and the original critical tasks. To ob- tain the posterior P* (0) (A))) M")
over the augmented critical tasks, we solve the optimization problem via gradient ascent. Specifically, we parameterize Po(0) as a Gaussian distribution whose parameter o (p. Σ) includes a mean vector 1l and covariance matrix Σ ooo We reparameterize 0 as 0 = / + a where Ç (O 7) denotes a standard Gaussian distribution and e is component-wise multiplication. The gradient of the lower-level optimization problem
Pc- VoL(O, (7, (Ax) to estimate the gradient: Is Ne E== VaOg VoL(0,, (T," (A.))
(T7)) (T"""""N""" and and we can use NS samples Çj (NO,
Ji=1
N
Not (T,")
N N
where
Ji=1
0j = μ + o Gz. We include the detailed expressions of the gradient in Appendix
G.1 Recall
that the posterior distribution PS (0((T"") Ji=1) (TT)) is obtained by marginalizing P* [T, (A,) (A
over {= Therefore, we sample N sets of mixing coefficients { {fA ) N 6==1 and compute
the posterior distribution Pa (-117, (A HE) corresponding each set. The posterior under the NY E ('(0 (1T)
original critical tasks can be estimated by (*(HTT
Upper-level optimization (line 4 We use gradient ascent to solve the upper-level problem. In order compute the the hyper-gradient i.e., the gradient of the conditional mutual information 4 w.r.t. Φx, we use a Gaussian distribution to parameterize Pox. (A) where the distribution parameter Φ (Mx, σx) includes a mean μ^ and a standard deviation Øx. Therefore, we can reparameterize each sample X
from Pox (A) via X pa + OxLi. where Čr.5 ~~(00,))
Lemma 1. Suppose we reparameterize the mixing coefficient X, via X M4 + OXLE, the hyper-
ENG. Vox ((A," )=l) 1 NG Vo, (12, (129)19 poni IE (12 (
gradient can be estimated by 96x
NO Σラ= (10 parr !
where Vo. (1A9 1-)
- -[V Ep (0) [L( (T:- (A (TM (0) (T0, (T:-0 () (T, Ji=1 (" }
2e E...,
We include the expression of all the gradients in Appendix G We solve the upper-level problem in 6) via gradient ascent Φx,4+1 = Φx + BBøx.x where 3 is the step size.
4.3 THEORETICAL. ANALYSIS
This part shows that (i) Algorithm converges at the rate of O(L/VK), (i) the learned augmentation increases the task information of the critical tasks stored in the meta-policy, and (iii) the generaliza- tion over the task distribution improves after the augmentation. We start with the assumption:
Assumption 1. The parameterized mela-policy ne satisfies the sollowing: [[ log (a(as)) s Co and 2 log Te (a((a))]] s Co for any (s, a) E x A where Cg and Og are positive constants.
Assumption assumes that the parameterized log-policy log no is Co-Lipschitz continuous and Co- smooth w.r.t. the parameter 6, which is standard assumption in RL (Liu & Zhu 2022 Kumar et al.,
2023 Liu & Zhu 2023 Agarwal et al. 2021 Liu Zhu 2024a b) Liu et al. 2025a).
Theorem 1. Suppose Assumption 1 holds and choose the step size B
V where
Ć₁ is positive constant whose derivation is in Appendix H then Algorithm
1
converges:
1 K-
>K=1
|V,, (0; (T Po (()) (T 112 < O(IVKK).
R
Theorem guarantees that Algorithm I converges the rate of ((1/VK)), We next show that the learned task augmentation stores more information in the meta-parameter:
Theorem 2. Suppose Assumption holds and 3 < 2.
TT' then the output Pax. (A) of Algorithm
satisfies I(0; (T. ~ Pox. (A)) NTT)) > 0.
Theorem 2 guarantees that the augmented critical tasks store additional information in the meta- parameter. Moreover, Appendix guarantees that the task information of the non-critical tasks stored in the meta-parameter does not change even if the stored task information of the critical tasks
7

---

Published as a conference paper at ICLR 2026
increases. We next quantify the generalization improvement of the learned augmentation Pøs,x x (2). In particular, we first demonstrate that the learned augmentation imposes a quadratic regulariza- tion on the meta-parameter in Lemma 2 and then guarantee that the generalization over the task distribution P(T) improves.
To reason about the generalization, consider the following softmax parameterized meta-policy c² ((e,0)
To(als) Eo'eg e" Menaa) where f(s a) is feature vector. This policy parameterization is widely
adopted in RL (Sutton al., 1999 Kakade 2001 Peters & Schaal (008). We consider MAML
(Finn et al. [217 Fallah et al. 2021 as the algorithm to compute the task-specific adaptation n.
and the task task-specific adaptation is also softmax parameterized.
Lemma 2. The second-order approximation of the meta-objective 33) after the task augmenta- tion can be expressed as: Ex.-Pos,K [L(0, (N))) (T,"")""" N""N""]] x L(0, (7.1) )
TT( Neri ENT H where H. is positive definite matrix whose expression is in Appendix K
Lemma shows that the augmented meta-objective (3) imposes quadratic regularization the original meta-objective (T. Since we aim to maximize the meta-objective, this negative quadratic regularization reduces the solution space and thus lead to better generalization.
To study the generalization property of this regularization, following (Zhang & Deng. 2021 Yao ctt al. 202I), we consider the following softmax policy class that closely related to the dual prob- lem of the regularization: FG = TTe 0 (Ei-eer) [M.)) S 7}. To quantify the improvement of generalization, we denote the generalization gap by ((F²) & L(O.(T,")
EE, Ti).
The following theorem validates improvement of generalization:
ot stow>
Theorem 3. Suppose the policy is softmax parameterized (i.e., ne(a|s) Edlen Mowwy where the feature vector f(s a) is wiiee-ifferentiable and bounded for any (s,a) E S A then with 7 probability at least 1 - 8, the generalization gap satisfies (9( S O1V No + / /1201/00)
Nor
the quadratic regularization (i.e., AT( New Σ Naii H () imposed by the i=1
According to Lemma learned task augmentation encourages a smaller 5. Therefore, according to Theorem 3 the learned task augmentation will lead to smaller generalization gap and thus improve generalization.
(
EXPERIMENT
This section uses two real-world experiments, two MuloCo exper- iments, and a Meta-World experiment (Appendix M.9) to validate the effectiveness of Algorithm (XMRL). Note that XMRL is to im- prove the performance of existing MRL algorithms, rather than being a standalone MRL algorithm. Accordingly, our primary comparisons against other MRL improvement methods, rather than against MRI. algorithms themselves. For completeness, we report compar- isons with some MRL algorithms in Appendix M.9 We use MAML as the base MRL algorithm that our method and the baselines aim to Figure 1: Drone navigation improve. We introduce three baselines that improve MRL generalization: (1) Task weighting (TW)
(Cai et al. 2020): This method computes the weighted meta- meta-policy To. TT* (w). (2) Meta augmentation
(MA) Yao al. 2021): This method uses a predefined distribution of X to mix the data of each training task to improve generalization. (3) Meta regularization (MR) (Wang et al., 2023): This method adds quadratic regularization to the upper level and inverted regularization to the lower level to improve generalization. Combining the base MRL algorithm with each improvement method results in four complete methods: MAML+XMRL, MAML+TW, MAML+MA, and MAML+MR. Note that method requires additional interactions with the environment to generate augmented samples. For a fair comparison, the baselines use the same amount of samples.
Experiment I: Drone navigation with obstacles. We conduct a navigation experiment (Figure on an AR.rrone 2.0 where the drone (yellow bounding box) navigates to the goal (green bounding box) while avoiding the obstacle (red bounding box). We motion capture system "Vicon" to
8

---

Published as a conference paper at ICLR 2026
Table Experiment results.
Method
MAML
MAML+XMRL
MAML+TW
MAML+MA
MAML+MR
Drone
0.87 + 0.01
0.97 + 0.01
0.87
0.91 = 0.02
0.91 0.02
Stock Market
359.13 + 18.63
121.13 12.11
362.07 + 14.21
389.17 12.66
362.53 14.27
HalfCheetah
-68.89 + 4.36
-11667 4.35
-65.14 4.26
-63.49 4.07 4.07
-61.15
Ant
100.64 3.63
119.15 10.15:::00 4.02
99.92 + 4.56
106.44 4.55
104.15 = 4.74
record the location of the drone. For different navigation tasks, we change the locations of the goal and obstacle. We success rate (i.e., the rate of successfully reaching the goal and avoiding the obstacle) over randomly generated test tasks as the metric to evaluate generalization performance. We record the mean and standard deviation of success rate in the second row in Table I
We sample 50 training tasks to learn a meta-policy. We then sample 10 tasks and find the top 3 tasks where the meta-policy adapts with the worst perfor- mance as poorly adapted tasks. In Figure 2 the red points and orange points the goals of the 50 training tasks. The green points the goals of the top 3 poorly adapted test tasks. The five orange points are the identified critical training tasks. We can see that the green points are far from the red points and thus are poorly adapted. The identified critical tasks are the training tasks whose goals are closest to the goals the poorly adapted test tasks.
Experiment II: Stock market. RL to train stock trading agent has been widely studied in AI for finance (Deng et al. 2016). We use the real-world alization
Figure 2: Task visu-
data of 30 constituent stocks in Dow Jones Industrial Average from 2021-01-01 to 2022-01-01. We use "FinRL" (Liu et al. 202T) to configure the real-world stock data into an MDP environment. include the details in Appendix M.5 and the results of cumulative reward in the third row in Table
Experiment III: MuJoCo. We consider the target velocity problem (Finn et al. 2017) for two MuJoCo robots: HalfCheetah and Ant. In particular, the robots aim to maintain a target velocity in each task, and the target velocity of different tasks is different. We include the details Appendix M.6land the results of cumulative reward in the fourth and fifth rows in Table
Table shows that our proposed method can significantly improve the generalization of MAML and outperform the other three baselines. For example, our method improves MAML by 35% for HalfCheetah while the baselines' improvements are less than 15%.
(a) Drone
Stock Market
(c) HalfCheetah
(d) Ant
Figure 3: Ablation study on the number of critical tasks.
The ablation study on the number of critical tasks Neri We evaluate how the number of critical tasks Nen affects the generalization performance of our method. We vary the number critical tasks and plot the corresponding generalization performance in Figure 3 The x-axis is the percentage of critical tasks in training tasks. Figure 3 shows that the optimal percentage of critical tasks 10% for drone, 30% for Stock Market, and T0% for HalfCheetah and Ant. Moreover, the generalization performance becomes worse if the number of critical tasks is too large. The reason is that at this time, the critical tasks can include training tasks that are not helpful to improve poorly adapted tasks. we still include those non-helpful tasks as critical tasks, those non-helpful tasks may prevent the meta- meta-pplicy from generalizing well tasks similar to the poorly adapted tasks. Note that even augmenting the "false positives" tasks can lead to a decline of generalization performance, the generalization performance is still better than MAMI. where tasks augmented.

---

Published as a conference paper at ICLR 2026
Minor degradation very few tasks. We conduct evaluation to separately report (i) the perfor- mance on the original poorly adapted tasks, (ii) the performance on the original non-poorly adapted tasks, (iii) the percentage of tasks with degradation in the original non-poon non-poorly adapted tasks, and (iv) performance drop for degraded tasks. For example, if the performance of a non-poorly adapted task drops from 100 to 90, then the performance drop is 10%. We report the average performance drop over all degraded tasks in Table 2 The results demonstrate that (i) the average performance on the poorly adapted tasks significantly improves, (ii) the average performance on the non-poorly adapted tasks does not degrade, (iii) only a very small portion (less or equal to 5%) of non non-poorly adapted tasks degrade, (iv) even for degraded tasks, the performance drop is minor (less than 4%).
Table 2: Ablation performance improvement and performance degradation
MAML.
Our method (MAML+XMRL)
poorly adapted tasks
0.55
0.93
non-poorly adapted tasks
0.95
0.98
Drone
percentage of degradation
N/A
0%
performance drop for degraded tasks
N/A
N/A
poorly adapted tasks non-poorly adapted tasks
71.05
381.33
431.15
431.08
Stock market
percentage of degradation
N/A
5%
performance drop for degraded tasks
N/A
3.8%
poorly adapted tasks non-poorly adapted tasks
162.09
-55.00
-45.59
-42.10
HalfCheetah
percentage of degradation
N/A
2.5%
performance drop for degraded tasks
N/A
2.7%
poorly adapted tasks non-poorly adapted tasks
39.68
99.67
115.88
124.02
Ant
percentage of degradation performance drop for degraded tasks
N/A
5%
N/A
2.0%
The ablation study on augmentation method. We aim to learn an optimal augmentation by op-
timizing this distribution P(A) to maximize the conditional mutual information (4). To show the effectiveness of our method, we include an ablation study in Appendix M.7 where we compare to method that uses the predefined distribution (Wang et al. 20200) to augment the critical tasks. The results in Appendix M.7 demonstrate that our method improves generalization better.
Evaluation of the explanation. We evaluate the fidelity and usefulness our explanation. Fidelity is (Guo et al., 2021b Cheng et al. 2024) to evaluate the correctness of the explanation. The fidelity our setting means whether the identified critical tasks {T."*) Ji=1 are indeed the most important training tasks achieve high cumulative reward on the poorly adapted tasks (T,P00r ate fidelity, we train meta-policy over the critical tasks and compare its performance on the poorly adapted tasks with a meta-policy trained on Neri nample training tasks. The usefulness means whether our explanation helps improve generalization. To evaluate the usefulness, domly pick Neri training tasks and use our augmentation method to augment these Neni training tasks to train meta-policy. We compare the generalization performance of this meta-polis with XMRL. The results in Appendix M.8 show that our explanation has high fidelity and usefulness.
N N To evalu-
ran-
CONCLUSION
This paper proposes to leverage explanation to improve generalization of MRL. The proposed method has two parts where the first part explains why the learned meta-policy does not adapt well to certain tasks by identifying the critical training tasks that the meta-policy does not pay enough attention to, and the second part formulates a bi-level optimization problem to learn how to augment the critical tasks such that the meta-policy can best pay attention to the critical tasks. We theo- retically guarantee that the learned augmentation can improve generalization over the whole task distribution. Experimental results validate that our method improves MRI.
10

---

Published as a conference paper at ICLR 2026
7 ACKNOWLEDGEMENTS
This work is partially supported by the National Science Foundation through grants ECCS 1846706 and ECCS 2140175. We would like to thank the reviewers for their insightful and constructive suggestions.
REFERENCES
Alckh Agarwal, Sham M Kakade, Jason D Lee, and Gaurav Mahajan. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research. 22(98))1-76, 2021.
Dan Amir and Ofra Amir. Highlights: Summarizing agent behavior to people. In International Conference on Autonomous Agents and MultiAgent Systems, pp. 1168-1176, 2018
Akanksha Atrey, Kaleigh Clary, and David Jensen. Exploratory not explanatory: Counterfactual analysis of saliency maps for deep reinforcement learning. International Conference on Learn- ing Representations, 2019.
Peter I. Bartlett and Shahar Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, 2002.
Osbert Bastani, Yewen Pu, and Armando Solar-Lezama, Verifiable reinforcement learning via policy extraction. Advances in Neural Information Processing Systems, 31:2499-2500 2018.
Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shi- mon Whiteson. A survey of meta-reinfforeement learning. arXiv preprint arXiv:2301.88026 2023.
Tom Bewley and Jonathan Lawry. Tripletree: A versatile interpretable representation of black box agents and their environments. In AAAI Conference on Artificial Intelligence, volume 35, pp- 11415- 11422, 2021.
Diana Cai, Rishit Sheth, Lester Mackey, and Nicolo Fusi. Weighted meta-learnin, arXiv preprint arXiv:2003.00 09466,, 2020.
Rich Caruana, Hooshang Kangarloo, John David Dionisio, Usha Sinha, and David Johnson. Case- based explanation of non-case-based learning methods. In American Medical Informatics Asso- ciation Symposium, pp. 212, 1999.
Yizhou Chen, Shizhuo Zhang, and Bryan Kian Hsiang Low. Near-optimal task selection for meta- learning with mutual information and online variational bayesian unlearning. In International Conference on Artificial Intelligence and Statisties, pp.
2022.
Zelei Cheng, Xian Wu, Jiahao Yu, Wenhai Sun, Wenbo Guo, and Xinyu Xing. StateMask: Explain- ing deep reinforcement learning through state mask. Advances Neural Information Processing Systems, 36, 2024.
Jonathan Crabbé, Zhaozhi Qian, Fergus Imrie, and Mihaela van der Schaar. Explaining latent rep- resentations with a corpus of examples. Advances in Neural Information Processing Systems, 34: 12154-12166, 2021.
Yue Deng, Feng Bao, Youyong Kong, Zhiquan Ren, and Qionghai Dai. Deep direct reinforcement learning for financial signal representation and trading. IEEE Transactions on Neural Nenworks and Learning Systems, 28(3):653-664 2016.
Guneet Singh Dhillon, Pratik Chaudhari, Avinash Ravichandran, and Stefano Soatto. A baseline for few-shot image classification. In International Conference Learning Representations, 2019.
Yan Duan, John Schulman, Xi Chen, Peter L Bartlett, Ilya Sutskever, and Pieter Abbeel. RL?: Fast reinforcement learning via slow reinforcement learning. arXiv preprint arXiv: 1.0279,, 2016.

---

Published as a conference paper at ICLR 2026
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of gradient- based model-agnostic meta-learning algorithms. In International Conference on Artificial Intelli- gence and Statistics, pp. 1082-1092, 2020.
Alireza Fallah, Kristian Georgiev Aryan Mokhtari, and Asuman Ozdaglar. On the convergence theory of debiased model-agnostic reinforrrrrr learning. Advances in Neural Information Processing Systems, 34:3096-3107, 2021.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leaming for fast adaptation of deep networks. In International Conference on Machine Learning, pp. 1126-11333 2017.
Jerome H Friedman and Bogdan E Popescu. Predictive learning via rule ensembles. The Annals of Applied Statistics, pp. 916-954, 2008
Ruohao Guo, Wei Xu, and Alan Ritter. Meta-tuning Ilms leverage lexical knowledge for gener- alizable language style understanding. In Annual Meeting of the Association for Computational Linguistics, pp. 13708-13738 2024.
Ruohao Guo, Afshin Oroojlooy, Roshan Sridhar, Miguel Ballesteros, Alan Ritter, and Dan Roth. Tree-based dialogue reinforced policy optimization for red-teaming attacks. arXiv preprint arXiv:2510.02286. 2025.
Sihang Guo, Ruohan Zhang, Bo Liu, Yifeng Zhu, Dana Ballard, Mary Hayhoe, and Peter Stone. Machine versus human attention in deep reinforcement learning tasks. Advances in Neural Infor- mation Processing Systems, 34:25370-25385, 2021a.
Wenbo Guo, Xian Wu, Usmann Khan, and Xinyu Xing. Edge: Explaining deep reinforcement learning policies. Advances in Neural Information Processing Systems, 34: 34:12222-12236, 2021b.
Mingyi Hong, Hoi-To Wai, Zhaoran Wang, and Zhuoran Yang. A two-timescale stochastic algorithm framework for bilevel optimization: Complexity analysis and application to actor-critic. SIAM Journal on Optimization, 33(1):147-180, 2023.
Hongrong Huang and Juergen Sturm. Tum simulator.
ROS package http://wiki, ros.
sigulatrs,,,,,or,, 2014.
Matthew T Jackson, Minqi Jiang, Jack Parker-Holder, Risto Vuorio, Chris Lu, Greg Farquhar, Shi- mon Whiteson, and Jakob Foerster. Discovering general reinforcement learning algorithms with adversarial environment design. Advances in Neural Information Processing Systems, 36:79980- 79998, 2023.
Minqi Jiang, Michael Dennis, Jack Parker-Holder, Jakob Foerster, Edward Grefenstette, and Tim Rocktäschel. Replay-guided adversarial environment design. Advances in Neural Information Processing Systems, 18:118811897, 2021.
Zoe Juozapaitis, Anurag Koul, Alan Fern, Martin Erwig, and Finale Doshi-Velez Explainable rein- forcement learning via reward decomposition. In IJCAI/ECAI Workshop on Explainable Artificial Intelligence, 2019.
Sham Kakade, natural policy gradient. Advances in Neural Information Processing Systems, 14:
1531-1538, 2001. 2001.
Zhaolu Kang, Junhao Gong, Jiaxu Yan, Wanke Xia, Yian Wang, Ziwen Wang, Huaxuan Ding, Zhuo Cheng, Wenhao Cao, Zhiyuan Feng, et al. Hssbench: Benchmarking humanities and social sciences ability for multimodal large language models. arXiv preprint arXiv:2506.03922, 2025.
Zong Ke, Yuqing Cao, Zhenrui Chen, Yuchen Yin. Shouchao He, and Yu Cheng. Early warning of cryptocurrency reversal risks via multi-source data. Finance Research Letters, pp. 107890, 2025.
Harshat Kumar, Alec Koppel, and Alejandro Ribeiro. On the sample complexity of actor-critic method for reinforcement learning with function approximation. Machine Learning, pp. 1-35, 2023.
12

---

Published as a conference paper at ICLR 2026
Guangchen Lan, Huseyin A Inan, Sahar Abdelnabi, Janardhan Kulkarni, Lukas Wutschitz, Reza Shokri, Christopher G Brimton, and Robert Sim. Contextual integrity LLMs via reasoning and reinforcement learning. In Annual Conference on Neural Information Processing Systems, 2025,
Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. Rein- forcement learning with augmented data. Advances in Neural Information Processing Systems, 33:19884- 19895, 2020.
Suyoung Lee, Myungsik Cho, and Youngchul Sung. Parameterizing non-parametric meta- reinforcement learning tasks via subtask decomposition. Advances in Neural Information Pro- cessing Systems, 36:43356 43383,, 2023.
Zhengxian Lin, Kin-Ho Lam, and Alan Fern. Contrastive explanations for reinforcement learning via embedded self predictions. In International Conference on Learning Representations, 2020.
Guiliang Liu, Sheng Xu, Shicheng Liu, Ashish Gaurav, Sriram Ganapathi Subramanian, and Pascal Poupart. A comprehensive survey on inverse constrained reinforcement learning: Definitions, progress and challenges. Transactions on Machine Learning Research, 2025a.
Hao Liu, Richard Socher, and Caiming Xiong. Taming MAML: Efficient unbiased meta- reinforcement learning. In International Conference on Machine Learning, pp. 4061-4071, 2019.
Shicheng Liu and Minghui Zhu. Distributed inverse constrained reinforcement learning for multi- agent systems. Advances in Neural Information Processing Systems,
2022.
Shicheng Liu and Minghui Zhu. Meta inverse constrained reinforcement learning: Convergence guarantee and generalization analysis. In International Conference on Learning Representations, 2023.
Shicheng Liu and Minghui Zhu. Learning multi-agent behaviors from distributed and streaming demonstrations. Advances in Neural Information Processing Systems, 36, 2024a.
Shicheng Liu and Minghui Zhu. In-trajectory inverse reinforcement learning: Learn incrementally from an ongoing trajectory. arXiv preprint 15601222, 2024b.
Shicheng Liu and Minghui Zhu. Utility: Utilizing explainable reinforcement learning to improve reinforcement learning. In International Conference Learning Representations, 2025.
Shicheng Liu, Siyuan Xu, Wenjie Qiu, Hangfan Zhang, and Minghui Zhu. Explainable reinforce- ment learning from human feedback to improve alignment. Advances in Neural Information Processing Systems, 2025b.
Xiao- Yang Liu, Hongyang Yang, Jiechao Gao, and Christina Dan Wang. Finrl: Deep reinforcement learning framework to automate trading in quantitative finance. ACM International Conference on Al in Finance, pp. 1-9, 2021.
Ricardo Luna Gutierrez and Matteo Leonetti. Information-theoreti task selection for meta- reinforcement learning. Advances in Neural Information Processing Systems, 33:20532-20542, 2020.
Cuong C Nguyen, Thanh-Toan Do, and Gustavo Carneiro. Probabilistic task modelling for meta- learning. In Uncertainty in Artificial Intelligence, pp. 781-791, 2021.
Cuong C Nguyen, Thanh-Tour Do, and Gustavo Carneiro. Task weighting in meta-learning with trajectory optimisation. Transactions on Machine Learning Research, 2023.
Kaichen Ouyang, Zong Ke, shengwei Fu, Lingjie Liu, Puning Zhao, and Dayu Hu. Learn from global correlations: Enhancing evolutionary algorithm via spectral gnn. arXiv preprint arXiv:2412.17629, 2024.
Jan Peters and Stefan Schaal. Natural actor-critie. Neurocomputing, 71(7-9):1 180- 1190, 2008
Nikaash Puri, Sukriti Verma, Piyush Gupta, Dhruv Kayastha, Shripad Deshmukh, Balaji Krishna- murthy, and Sameer Singh. Explain your move: Understanding agent actions using specific and relevant feature attribution. In International Conference on Learning Representations, 2019.
13

---

Published as a conference paper at ICLR 2026
Guanren Qiao, Guiliang Liu, Pascal Poupart, and Zhiqiang Xu. Multi-modal inverse constrained reinforcement learning from a mixture of demonstrations. Advances in Neural Information Pro- cessing Systems, 36:60384-60333 60396, 2023.
Janarthanan Rajendran, Alexander Irpan, and Eric Jang. Meta-learning requires meta-auugmentation. Advances in Neural Information Processing Systems, 33:5705-5715, 2020.
Aravind Rajeswaran, Chelsea Finn, Sham M Kakade, and Sergey Levine. Meta-learning with im- plicit gradients. Advances in Neural Information Processing Systems, 2019.
Kate Rakelly, Aurick Zhou, Chelsea Finn, Sergey Levine, and Deirdre Quillen. Efficient off-policy
meta-reinnorreemnnt learning via probabilistic context variables. In International Conference Machine Learning, pp. 5331-5340, 2019.
Yael Septon, Tobias Huber, Elisabeth André, and Ofra Amir. Integrating policy summaries with
reward decomposition for explaining reinforcement learning agents. In International Conference on Practical Applications of Agents and Multi-Agent Systems, pp. 320-332, 2023.
Gresa Shala, André Biedenkapp, Pierre Krack, Florian Walter, and Josif Grabocka. Efficient cross- episode RL. In International Conference on Learning Representations, 2025.
Xinyue Shao, Hongzhi Wang. Xiao Zhu, and Feng Xiong. FIND: Explainable framework for meta- learning. arXiv preprini arXiv:2205. 10362, 2022.
Xinyue Shao, Hongzhi Wang, Xiao Zhu, Feng Xiong, Tianyu Mu, and Yan Zhang. EFFECT: Ex- plainable framework for meta teta-leanning in automatic classification algorithm selection. Informa- fion Sciences, 622:211 234, 2023.
Bradly C Stadie, Ge Yang, Rein Houthooft, Xi Chen, Yan Duan, Yuhuai Wu, Pieter Abbeel, and Ilya Sutskever. Some considerations on learning to explore via meta-reinforcement learning. Advances in Neural Information Processing Systems, pp. 9280-9290, 2018.
Hao Sun, Alihan Hüyük, Daniel Jarrett, and Mihaela van der Schaar. Accountability offline reinforcement learning: Explaining decisions with a corpus of examples. Advances in Neural Information Processing Systems, 36, 2024.
Richard Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient meth- ods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12:1057-1063, 1999.
Abhinay Verma, Vijayaraghavan Murali, Rishabh Singh, Pushmeet Kohli, and Swarat Chaudhuri. Programmatically interpretable reinforcement learning. In International Conference on Machine Learming, pp. 5045-5054, 2018.
Kaixin Wang, Bingyi Kang, Jie Shao, and Jiashi Feng. Improving generalization in reinforcement learning with mixture regularization. Advances Neural information Processing Systems, 33: 7968-7978, 2020.
Lianzhe Wang, Shiji Zhou, Shanghang Zhang, Xu Chu, Heng Chang, and Wenwu Zhu. Improving generalization of meta-learning with inverted regularization at inver-level. IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 7826-7835 2023.
Xiaojia Wang, Anxing Bao, Pengfei Pan, Hai Wan, and Sheng Xu. A dynamic adaptive selection method for multi-objective flexible job shop scheduling schemes with interval linguistic informa- tion. Journal of Engineering Design, pp. 1-25, 2025.
Katarzyna Wožnica and Przemyslaw Biecek. Towards explainable meta-learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 505-520, 2021.
Aaron D Wyner. A definition of conditional mutual information for arbitrary ensembles. Information and Control, 38(1):51-59) 1978.
Zhongwen Xu, Hado van Hasselt, and David Silver. Meta-gradient reinforcement learning. Ad- vances in Neural Information Processing Systems, 11:2402-2413, 2018.
14

---

Published as a conference paper at ICLR 2026
Huaxiu Yao, Long- Kai Huang, Linjun Zhang, Ying Wei, Li Tian, James Zou, Junzhou Huang, et al. Improving generalization in meta-learning via task augmentation. In International Conference on Machine Learning, pp. 11887-11897, 2021.
Mingzhang Yin, George Tucker, Mingyuan Zhou, Sergey Levine, and Chelsea Finn. Meta-learning without memorization. In International Conference Learning Representations, 2019.
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-worid, A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on Robot Learning, pp. 1094-1100, 2020.
Donglin Zhan and James Anderson. Data-efficient and robust task selection for meta-learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6056-8065, 2024.
Hongyi Zhang, Moustapha Cisse, Yann N. dauphin, and David Lipez-Paz. mixup: Beyond empirical risk minimization. In International Conference on Learning Representations, 2018.
Linjun Zhang and Zhun Deng. How does mixup help with robustness and generalization? In International Conference Learning Representations, 2021.
Tao Zhang. Episodic-free task selection for few-shot learning. arXiv preprint arXiv:2402.00092, 2024.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning, In International Conference on Machine Learning. pp. 7693-7702, 2019a
Luisa Zintgraf, Kyriacos Shiarlis, Maximilian Igl, Sebastian Schulze, Yarin Gal, Katja Hofmann, and Shimon Whiteson. Varibad: A very good method for bayes-adaptive deep via meta- learning. In International Conference on Learning Representations, 2019b.
A RELATED WORKS
Metaa-Reinforcement learning. MRL has two major categories: optimization- MRL and black-box (or context-based) MRL. Optimization-basel MRL (Finn al. 2017 Liu et al. 2019 Stadie et al. 2018) usually includes a meta-algorithm and an adaptation algorithm. The meta- algorithm learns meta-prior (e.g., meta-policy) which is not specialized for each task. The adap- tation algorithm uses data of a specific task to specialize the meta-prior to achieve high cumulative reward on this specific task. Black-box MRL (Duan et al., 2016 Rakelly et al. 2019 Zintgraf al. 2019b) typically learns end-to-end model that contains specialized knowledge for different tasks in the task distribution. The data of a task is used to indicate the task so that the end-to-end model can directly specialize to this specific task. Note that the method proposed in this paper is not classi- fied as a MRL method. Instead, we aim to develop a method that can improve MRL and this method can be applied to different MRL algorithms.
Meta-learning generalization improvement. There are three major ways to improve meta-learning generalization: task weighting, regularization, and meta-augmmentttion. Task weighting (Nguyen et al. 2023 Yao et al. 2021 Cai et al. 2020) proposes to re-weight the training tasks or reshape the training task distribution improve generalization. However, Yaoo et al. 202 Cai et al. 2020) require an additional target task set to guide how to weight the training tasks or reshape the training task distribution, and thus the learned meta-prior can be biased towards the target task set and may not adapt well to other tasks that are different from the target task set. Regularization- -basd methods are also used improve generalization where (Wang et al., 2023) proposes to add ordinary regularization to the upper level and inverted regularization to the lower level, and (Yin et al.) 2019) imposes regularization to prevent memorization overfitting. The most relevant technique to our paper is meta-augmentation which augments the data and train on the augmented data to improve generalization. Specifically, (Rajendran et al., 2020) proposes to add noise to the data and (Yao) et al. 2021) proposes to mix data and shuffle the channels in the hidden layers. The augmentation method has also been used in RL (Wang et al.) 20200] Laskin et al., 2020) to improve generalization by generating augmented data. These augmentation methods use predetined rules to provide feasible
15

---

Published as a conference paper at ICLR 2026
augmentations. In contrast, our paper formulates bilevel optimization problem to learn how to best augment the critical tasks.
Explainable reinforcement learning. While lacks research works explainable MRL, explain- able RL (XRL) has been extensively studied to explain the decision making of the RL agents, in-
cluding learning an interpretable policy (Bastani et al. 2018 Bewley & Lawry 2021 Verma al.
2018), pinpointing regions in the observations that are critical for choosing certain actions
(Atrey
al. 2019 Guo al. 202 la Puri et al.) 2019), and reward decomposition (Juozapaitis al.
2019
Lin et al. 2020 Septon et al. 2023). The most relevant XRL method to our explanation is to identify
the critical states that are influential to the cumulative reward as an explanation (Guo al. 2021b
Cheng et al.] [024 Amir & Amir 2018) where they respectively use an RNN, masks, and a self-
proposed rule to find critical states. contrast, we formulate bilevel optimization problem to learn
weight vector that indicates critical tasks.
Explainable meta-learning. There are three works on explainable leatnnn where (Wožnica & Biecek] 2021) proposes to learn important features that lead to a specific meta model decision using Friedman's 's H-statistic (Friedman & Popescul 2008), and (Shao et al.) 2022; 2023) use structural causal model to model the causal relations between the features and the model decision. While
these works explain why a decision is made, we explain why certain tasks are poorly adapted.
Task selection of meta learning. Task selection has been studied in meta-learning (Zhan & Ander- son 2024] Chen et al.] 2022) focus on task efficiency and propose to select subset of training tasks that are most representative informative so that training fewer tasks (the selected subset) can achieve performance comparable to training on the whole training tasks. In contrast, our method has different focus, i.e., it does not aim used fewer training tasks to achieve similar performance but aims to select and focus on critical tasks so as to improve performance. (Luna Gutierrez & Leonetti 2020 Zhang] 2024) propose to select training tasks such that the meta-prior can perform the best on a certain test task. However, they assume the ability to evaluate on test set. In contrast, our method perform well on a specific test set.
does not assume access test set and our goal
Unsupervised environment design. Unsupervised environment design (Jang et al. 202 Jackson al. 2023) adaptively and progessively generates more complex tasks for the agent to solve order to improve the agent's generalization ability. However, our paper does not generate or require new tasks to improve generalization. In contrast, we identify "critical tasks" from the existing training tasks and augment these critical tasks to improve generalization.
B ALGORITHM TO FIND THE CRITICAL TASKS
Recall from Section that we aim to learn a weight vector w by solving the problem (2) where each component w; of the weight vector captures the importance of the corresponding traiming task T. The higher the weight value Wi is, the more important the corresponding training task T, is. Therefore, the top Nea training tasks with highest weight values are the Neri critical tasks we aim to identify. The problem
(2) is as follows:
max (w), (T, NTL S.I. 0" (w) arg max Σ (F) W
We use Algorithm 2 to solve this problem where at each iteration k. we first solve the lower-level problem in (2) to get 0* (w)) and then solve the upper-level problem via gradient ascent.
Algorithm Identifying the critical tasks
Input: Training tasks {7. N" poorly adapted tasks (T,e Output: Learned weight vector wg.
Npeer Ji=1 and initial weight vector w..
1: for k= 0, 0. K do
2: Solve the lower-level problem via gradient ascent 0 = 0g + ag Σ wi V (Tt
3: Compute the hyper-gradient gwz, in Lemma 3 and update the weight wi+1 w + d.
4: end for
16

---

Published as a conference paper at ICLR 2026
Solve the lower-level problem. We use gradient ascent solve the lower-level problem where the gradient is Σ wi (n) and the expression VoJ, (T. can be found in Appendix
G.1
Solve the upper-level problem. solve the upper-level problem, we need to compute the hyper- gradient fw.
Lemma 3. The exper-gradient is:
9w
N
Atr wi
NDooe
V 2 WiVe
( (0" (w))))
Σ
Σ
VoJt
poor (mg
poor
(0 (w)))
where the derivation is in Appendix B.
In practice, proximation
we
do
not
have approximate
(wgg)
and thus
we
use
the
ap-
8
to
the
hyper-gradient
V. E
V V (T. (0=))
Ei wa 1 (m (OR)) 1=
Nooo
V Vo
(T
poor (0g))
We use K K-step gradient ascent WK+1 wE 0k9w to solve the problem (2) to get the learned weight wg- Each component wK,a captures the importance of the corresponding training task 7," We pick the top Noi training tasks with the highest weight value as the critical tasks.
B. PROOF OF LEMMA 3
Since (w) arg max; En (m))) then Vo Ei (T = 0. Take gradient w.r.t. w on both sides, we have that
N
N" Σ (.
wi (T (0") (WO
V
0,
49
N
N "E
(
V/
Σ (T.
Therefore, we have that
V L( (T. pour (im))
(0000-((( (ww VOL(O (T.uear (.
N
N Σ (x
(WrT( V
VoL(O' (o.err )TT
(T)
where (a) follows
C EXPRESSION OF THE AUGMENTED STATE-ACTION STATIONARY DISTRIBUTION
This section provides the expression of the augmented stationary state distribution p As and aug- mented stationary state-action distribution p" (..-). The expression of the augmented stationary state distribution is fT (5) A (8) + S I I()s + (1 - Ar)s' = 3)p ps) (0)p((( (s0)"sssddd (s')dsds' and the
expression of the augmented state-action stationary distribution is p du (s, = pmi (a)s).
D DERIVATION OF THE CONDITIONAL MUTUAL INFORMATION
T(0; (A, P(A)) A
17

---

Published as a conference paper at ICLR 2026
(a)
/ P(O, (4)) N
P(O, -
log
(d)) (A, = Ab) )
P( (((T7
(6)
PIO A = X:) ( M P( (A; = > P(( )
P(O. (T, (A, = {A)
log
(d)) (A, = -d)
PIO
(A, {m) Noi
(c) / PIOITT (Ai X:) ) 1=1 (7, P(LA: A+) P( )
P(O, (T. [A, =
(T," (d0) (A. = A 1T))
log
P(O
(d)
PIONTT (As = X,) / (7:"))))
P(fA, - - P((T
P(O, (T. (A. A,) 11 Noi
log
(do)(d{A | NMM Ad Nr }==)),
PIO
(A.
/ P(ONT (Ae X (7, ) A P( P( |
PTO (A;
log
(d0) A )
P(0 (T, |
P(ONT. (A. = A P(fA, = P(TT)(1
P(O (A; ))
log
(dd) X (d(T:"
PA ( (A))) log P(0 (T. (A))
[T,
J=
where (a) follows the definition of conditional mutual information (ynne 1978). (b) follows the
standard chain rule of probability P(A, B, C) P(A ())( (c) follows the fact that
P((T:" (A, {)) N (7, )==)) = P( [A, = m (explained in detail in Appendix E (d) fol-
lows the fact that f (( = ) ((T = > = f P(T. = { (((T))) A because (A, = A,) is determinisitically determined by T: and X, and (e) follows the fact that PA = A,) (T. )-)) = P( = ( because the
meta-parameter is trained on the augmented critical tasks (T, A=)
E DERIVATION OF THE POSTERIOR DISTRIBUTION
5
zagment
train
i distribution
Figure 4: How the distribution of is computed.
Figure shows how the distribution of the meta- 9 is computed. At the beginning, we have the original critical tasks (T." N (,, however, the distribution of 0 is not directly trained over the original critical tasks. Instead, we sample { from P()) and augment the original critical tasks to the augmented critical tasks (A: and train the distribution of 0 over the augmented critical tasks. Since 0 is directly trained on the augmented critical tasks (A,)) the posterior
18

---

Published as a conference paper at ICLR 2026
distribution of f given the augmented critical tasks can be computed by solving the distributional op-
timization problem: P* (' (A,) HE) arg max (0) (0, (7,Th (A:) ( N" N"")
(7.")
However, we cannot use the same way to formulate distributional optimization problem compute the posterior distribution (- (
I=e) because the meta parameere is not directly trained over
the original critical tasks (7, Ji=1 We can obtain this posterior distribution by marginalizing over (TxTh (A.) N:
[ PA
P" (" N
(T, = X.) 1T
PM
(A, (A+
- (((
(0)
P*(O1(TT (A=N)))))( (A, - (F
(2. N
(0 = P(
FA. (A) (A]))
where (a) follows the fact that (0) (Ai { 1=1 P" = 2 because directly trained on {T, (A, >.)) and (b) follows the fact that P((T:" (A, A ) = P(IA, = (g) and the reason is provided as follows. For simplicity, we first consider one task, e.g., the first critical task 7T We use A₁ to denote a random variable that follows the distribution of P()) and use λ to denote a specific value sampled from P(A), e.g..
=
A1 = 0.5. Therefore, (A, = t1 = 0.5) is specific task because a1 is specific value. Note that the critical task 71 is training task that already given, therefore, the probability of the (random) augmented task (A,) being the specific task (Ai = a = 0.5) is the probability of the specific value A = 0.5 being sampled from P(A). Therefore, P(T A = (T5 P(A1 = A1 = 0.5).
F PROOF OF LEMMA I
Recall from that
(0: (A, Pox (A)) (T: 1=))
(O1 (T.TT(()))) (A.))
E A, (1),6- log P* (0 (.
Since P (1A N
(0) is Gaussian distribution, we have that
T(0: (T.-t (A, ~ Pox
exp(- 1 (HASHET) D (t
Vioo (14 TO
=E,,0 log
exp( 1:000000000000 (E 10-00 (LA N
Ex:
V (a (((
exp(- (E
= E log
exp((- -0-00-0(((A)))
EE log Es.
Vilo ( ( ((AS))NT))
exp(-25TT
EçonV(0.1) Ex; log
V ((o T (1A))
19

---

Published as a conference paper at ICLR 2026
exp(--
- log EA,
((A9) To (
E', log
log Ex,
VIlo o o
V1(0 ((A9))))))))) ar ( (LA M
where (a) follows the fact that = μ (HA ( + o² o. Since we sample N sets
of mixing coefficients {(1)) LNE from Pax (A), the conditional mutual information can be esti- mated by
1(0;
(T ~ Pox (A)) (T:
N
N
NS log
Σ
log N
VIlo~(1>5 ) az (115) JEE
j=1 V (11 (16))1 ) To (12 (11S) (
/=1
Therefore, we can get the gradient:
Vos [(: (T. (A, ~ Pos (A))] (T NM 1i=1); {T,
NS j== Vo. o² ((,, N
N² Σ j=1 110 (1 25 ( I|
V o (() ((1))) (
EØ o² (12,) 2 (
li=1 |
NO
To get Vo, we know that 0- = arg max (0) [00 (T (X ) (T:T)))))) Jim1 }). there- fore, we have that VoLp,, (0) (0, (T, ( |
(T,t) N" Non
) = 0. Then we have that
أi=1
V Ep (0) (00 (T, | (T:1)
J==1 - - )
doy
Voga Ep, (0) [(0 (T. (A Nor (T, - ]
M==1
V Ep, (0) [(0 (T: N (T:th NM [] 2,.0"
VO. - V Ep pp. (0) [L( (T. (A )
yoi (T, N
V Ros Epg, (0) [L( (T: (2 M (T, N ~ )]
1==1
G
GRADIENTS
This section provides all the gradients needed in this paper.
G.1 META-GRADIENTS FOR MAJOR MRL METHODS
Recall the problem formulation of MRL follows where we omit the superscript for simplicity:
1 Σ (.)))()). s.t. (0) = Alg(TO. Ti). N
max L(0, (T.H 0
=
The meta- gradient is the gradient of the upper-level objective w.r.t. 6, i.e., VoL (T)HN......... The meta-gradient is different for different algorithms because different algorithms use different ways to compute the task specific adaptations (0). Here, we provide the meta-grade artaa-gradients for several major MRL algorithms, including MAML (Finn et al., [017]] Fallah et al.) (02T). iMAML (Rajeswaran)
et al. 2019), and context-based MRL (c.g., CAVIA (Zintgraf et al. 2019ah).
Lemma 4. The meta meau-igudsents for MAML, iMAML, and CAVIA are respectively:
N
VoL(O, (T)NN) = N
Σ I Vo. Vo.
(MAML)
20

---

Published as a conference paper at ICLR 2026
N N
(T.
+
V2 vy
(Vo (iTe).
(iMAML)
VOL(O, (T))
(T
(CAVIA)
N
where a is step size, 0, HaHaaoJiiaa Vg, (T(.) = ((aaa)="* " VV, log ne, (als) (),
(iTo)) = B( Eiso r' y'Vo (s [log To(a) (s. log (als)) +
Ve (( (log To (a)s)) Q? (s) A is hyper-parameter, 0f arg max v J / 2
0 P no( v)) is context-based baseed policy where v/," 1/ + a J. ( vo)) is the context.
Proof. MAML computes the task-specific adaptation via one-step gradient ascent. Specifically, suppose the task-specific adaptation is Te, = (0), and thus 0, 0 + (To.. Therefore, the meta-gradient is VoL(O, (T.) = $ EZ= (mo. = E (Vo0.) J. =
From (Fallah et al. 2021). J we can get that the policy gradient is Ve, (mTe)) = (((sa))p VV. log na (4,") (s, a)] and the Hessian (TTO) Biaa))ppo
N DE +
2'V [log Qa (s, a)](Va log (a(a))) +
V [log To (o(a|s) (s.,a))
iMAML solves the optimization problem to get the task-specific adaptation To; such that 0j arg max, (Tp) + 2 lle - 01 where À is hyper-parameter. Since 8 is the optimal parameter
of the problem maxy ( + Ill - 012, we know that VoJ)((T))) + Ae0; - 0) = 0. Take gradient w.r.t. A on both sides, we can get that (Va()) 2i (V + A(Vo6% - I) - Vodg = wh (((o))) Therefore, the meta-gradient is VoL(O, (T.) = N E VoJi(TT.) = (
1+
-= (Vll)) (oTT) = E. V ((oo)) V.
N
CAVIA learns context-based policy (a w²r) and uses MAML-like method to update W No + aV W. voo). Therefore, the meta-gradient is VoL(O, (T.WW)) 1 DM VoJi(To(-Is v))).
G.2 OTHER GRADIENTS
This part provides the expressions of V2 Ep,, (0) [[0, (T, (I." (1 (T,t (T and
Vec, (0) (T." Not (T." J= - needed in Lemma 1
Lemma 5. We have the following expressions:
Ep, (0) LLO, (T.TH (A D
(T, Jiml
"" }
Now
N A
-c-N((0.1) Nw
Jort ( (1 + +o o 1,,9))
Jo +o" <m1] +0)],)]]
V2o, Ers, Ers.(0)) (10, (T:
N [TTT) -N")) }
No
Σ No I
= =E Nw
p (sigr, azr) Vo,s log no, (B (0. 85))
Asg' + p Jonds (853', azf') Vo. log no, (āzz K3f (A, (8j, aa) A, (syr, )y)) days dajs'
where the expression of the second- order term V2 no Jac +o + c): Xx) can be found in Lemma
21

---

Published as a conference paper at ICLR 2026
Proof. Recall that x- """, 0 μ too and ~ ( I). Therefore, we have that
VoErs (0) 100 (7, (A M (7," |زー1
1 - }
Bc-.No.n)\VoL + +o o C: (T,00) (A57 ) (7: (T. )== NM }|,
Jim1
N
N= Nmn
= -EC-N(O.1
Vo
J.th ( (p* 0² o ,),
+
Σ
VoJ. + +o [[]]] (9)
Ni
Therefore, we can get the Hessian:
V Epg. Ep)-(0) (L, (T,
DAT
(T:t) ( }
N No
= -
1
a (1²
+o (), +
Ve of (u" +o .(II]
N
00
where the expression of the second-order term V V2. ( (n* +o Ar) can be found in Lemma Similarly, we can get that
oox ppp-(0) (10, (T,
2 Noi M==
( }}
Noi
No Noi
v oo. Jo (π. (H² + t o (), +
Σ ". +o -
E
No
N
-EC-N(O.1
Nir odh Ju
V
(π. (1² + + + +xxii]
Now we need to derive the expression of V (n;" + a² S),u + oxLi). Suppose we use MAML, and thus the first-order gradient V J. (π; (n" + +o* C),a + و2دد) (e, Hy + xx5i)l where 02 n² +
[I+ + VV2 V To
VoJ (m MA +x+)) and A 1²" to" of. Following the first-order MAML method in Fall lah et al. 2020). use the gradient V V. (T." (n" +o" (xx(() = VoJi(TO, MY +
To get the term V
Vopa Jem (mf (p- + +o 0), + xxS)), we derive Ve.s JTTe, MA +
V 0, Ji(TT, MA + Vol (0,1 Vo. log mo. (رح ورت Agyl]
Vo
e (s3s, agf Ve, log ne. (āر /33 A da da
Vox
p (83J', Ag5 Ve, log ne. (ā, 13 A day,
(EHHAS))ESAA
(eSH Vosds
VN pm ور Vo. lo (3, و A dagy
>s j(sy)aayy)EAA procis ($3z) a35') Ve.s log To, (az 1388 (o; - 8. Agy'
+ t ($j5, Oz8 Vo, log To, (azy 1538 (A, Th az) A, (sy az ))) day
where (a) follows the fact that VA, TT (gi5', 0.3') = 0 and V log to. (a,h (8,))) because they include indicator functions. Therefore, we have that
Voog Epp- [L0, (TeTH (A DAT (T.T JN -N"") }
Nen
= =ECNN(O.))
A3
I Jesragyeesa
pr (835, (33+)) (Vo.. log (155) - sy))
Nu
1=1
Agg + pTo (183"1 (853", (ij')) Vo. log no, (35) (4,51 (A55"((),) (z))aa)) A, 1 (syr ayy))) da do
22

---

Published as a conference paper at ICLR 2026
H PROOF OF THEOREM
This section first prove that the conditional mutual information [(0; (7,7 Pos, (A)) (an))1l(200 (7;" (T:"") (T) is Cy-Liischitz continuous and Cy-smooth where C₁ and C₁ are positive constants in Claim 1 and then prove that Algorithm converges at the rate of O(1/V
Claim 1. The conditional mutual information is Cy-Lipschitz continuous and Cy-smooth where C₁ and C₁ are positive constants.
Proof. From
we know that
((0:TT," Pox (A))
N
-Ex; log
log Ex,
(((o(({{())TT)) To o ((1 1
Vllo (( a ((25),
where A5 = = + 02h and G7 ~ NO, 1). Therefore, we can get the gradient
Vol T(0; (T: (); ~ Pe. (A) N
De o1 o² ((), (12 NoT
V 2 o² ((1S)AVO))
((A
B(aN(a.))
(10)
[ lo ((2 ji I]I
(100(12()
Now, we consider the Hessian
V2
(A. Pa, (A))
dxda
EE-N(O.1) or ((15)))))
V S) o" (L2, Nem
=Va
FE-N10.1) Vdl
||E ((^5) )= Nir
110 ((2S)
Ez-N(O.1)) V Vos ((15) ((
|ER-NiO.)lo (LA (.r))) IIl
Ei-N1O,) Voro" 0" (12)) (E (~ la
lo (125) Not 11) T V (11
| Ve lo ((1
Neri }]|
Neri
Ji=l Net
V Bxor o ((2 Not - -E--((O.))
((A))-11)
Val o ( Ji
(a ((2
TV Ve ((1))
| (15
|o (LA Mur ||3
(11)
From (10). we know that if we can lower bound ||o*| and upper bound
the norm of
(T,er)) Ji=) is bounded. From (1 we know that if the gradient Ve, [(;; (TTa ~ Pox ( N
lower bound ||a"|| and upper bound || o" and || V2ss20 o"Il. the norm of the Hessian IIV o0 [(0; (A; Po, (A)) Jilll is bounded. Note that A [0, 1] is bounded within compact set. Therefore, long as we can prove that o", Vora*. and V2,0, a" are con- tinuous in X, their norms both upper bounded and lower bounded. To show that a², Voxa", and V P2 o" continuous in X, we can show that they differentiable w.r.t. X. Since dx is differentiable w.r.t. A, we only need to show that a², V Vay o" and V V2l020 o" are differentiable w.r.t.
Φx This suffices to show that V Vd, a², V2 Pxdd a", and 3 exdios 0" exist.
From Lemma
I we know that V Voye a² exists and
V E2 p. (0) (L0, (T, (1 MAT
(T. No N Nori )]] Ji=l
V2 oo, Erg- (0) (40, (.. N N"
23

---

Published as a conference paper at ICLR 2026
Since log my smooth in 0 (Assumption ! we can see that I(0, (T, (A, ( } (,:म);;
also smooth in 8. Since 9 is smooth in o, L(0, (T. M (T,-) g. Similarly, we can derive
N - is also smooth in :==1
Vosi = V2 20 Ep Ep. (0, (L) (T, (A M (T," Ji - )]]
V
roOd Ep. ( (0) [L (T. (A ب (T I= N" >
V 2 00 Ep (0) [0 17, (A / ) (7. 1 [--"" }]]
V2 Ep,, (0) (0, (,, (A; M [ - -MT"
V Joroo Ep. (0) (0, ( - (7: 1i=1 - Nm1 )}
and similarly we can derive the expression of V Va o'. Therefore, we can see that ||o"||, VVo0""ll and |/V 2io, a²Il are both lower bounded and upper bounded, and thus there exists pos- itive constants C₁ and C such that IIV Φx T(O; (T, (A; Pax ( s C1 and [IV exox I(0; (T. ~ Po. (A)) N (7 RElI I C2.
For simplicity, we denote f(xx,)) = I(0; (A; ~ Por. (=) Nai (7:00)))))))))) Claim shows that f(xx,x) is Cy-smooth, therefore, we have that
f(oxa+)) 2 f(xx)) (V ((Ax)), )xa)
|1 $x,e11?
C,82
@
2
$ Vl f(xx S f((AAAt)) f(xx,)+ + C1C7B2
2
CIVK
([(x)+1)) - (((A=))))((()))))))))))))))))))) + C7
2
VK'
(1(0kk) - -((aoo))((()))) + C7 2
Or Or
VK"
where (a) follows the fact that Φx = Φ + Vorf((r.)) (b) follows the fact that o/ f(e))l < C, and (c) follows the fact that B = CiVK
I PROOF OF THEOREM 2
This section proves Theorem via two steps. Step (i) prove that 1 [0; (A: Pos. (A)) (T.ad) JEE)
monotonically increasing in Claim 2 Step (ii): we provide that
I(0: (X: Pos, (A))) > 0.
Claim 2. If 8 < &, the conditional mutual information is monotonically increasing, i.e. T(O; (A. ~ (A)) (T:" 2 T(0; (T. ~ P (A)) MMT), and is strictly increasing if e, I(e: (T: (A, P
> 0.
Proof. For simplicity, denote f(6x,)) = 1(8: (T.-0 (As & M (T. Therefore, we have that
(a) f(xx411) 2 ((xx)) + ((ox)) )x,))
- -11 s2all
G,53
f(oxx) +
(bs.(())))1
2
24

---

Published as a conference paper at ICLR 2026
- 2 - C132 [(
(12)
2
where (a) follows the fact that f(xx) is Cy-smooth (Claim 1 (b) follows the fact that @x,k+1 =
ΦA & f(0x,)). from (12), we can see that f(0x.++) 2 f(oxa), Moreover, f(AAA-1)
f(0xa) o. f(xxx)] 0.
From Claim 2 we know that [(;; (A, ~ Pox.k(A))) (7, 1 2 I(; (T.-h (Ae
(o)) (T,"") MT). The only situation where I(;; (T, (A: ~ K (A)) (T. Jim!
[(0; (7. (A, ~ Pox,ol (
is that Vo. [(0; (7, (dr ~ Po ) ( (i stationary point, which is of zero probability. Therefore, we know that
0, i.e.e the initialization
[(:: (T," ~ Po..K [A))]TT 1i=1) > [ (A. Pa).o (7, 1 1i=1). Since
conditional mutual information is always nonnegative (Wyner 1978). we know that ((; (7, (A.
Pos. (A)) E ) > [(;; (T, Po (a))
2 O.
J THE TASK INFORMATION OF THE NON-CRITICAL TASKS STORED IN THE META- PARAMETRR DOES NOT CHANGE AFTER THE TASK AUGMENTATION
This section shows that the task information of the non-critical tasks stored in the meta-parametee does not change after the task augmentation. In brief, we prove that the mutual information between the meta-parameter and the non-critical tasks remains unchanged even if the mutual information between the meta- parameareeeter and the critical tasks increases after task augmentation (i.e., the condi- tional mutual information is positive).
Suppose we augment the critical tasks (T,"H Fi-1 (T,"")) 1i-1- Note that the difference be- tween [T,00 Ji=1 Ji=1 and (T.-") Ji=1 is that they have different distributions, i.c., P((T.- (=)) and P([T." M Therefore, we use A to generally represent the critical tasks (either before augmen- tation or after augmentation), and use P(A = (T:" and P(A = (T)
respectively denote that A follows the distribution of [ and A follows the distribution of [ N We now quantify the change of the mutual information between the patamparreeter and the non-critical tasks
(7,1) N" N
1= - (0: (7," Fi=1
IT
Ji=1 (0: 1T NM "1T"" (7)
P(0, Ji=1 - (T.
P(O, (T.H Ji=1 (T
log
(0t N* [d( [d(T.--))))
P(O Lo P( No )1
/ P(O, (T." Ji=1 (T.T N
N" ~M
JE
P(0,
log
do (d((Tt 11" Ji=1
(d(TT"uJAA))))
[T )
N N
T Non
1=1
P(O N" (T N
N" N = P(0, (T." li=1
:T. J==l log
o. Nei
dB(d N "
P( Jiml
Nai
-f P(O, (T. No
P(0 (T. (T.1)N - (T
AT Noi
log
11=1
do (T
Fi=l [d{T"")
P(0) (T.ari) 1==1
( / POHTTT NY A)P([T)))NT N" No0
= )P(AA
PP(A
No)
(TO N"
Ji=I d" (dAA)
log
P(O|A)
P(O N" A)P(TT")) N - (T: (T )M
25

---

Published as a conference paper at ICLR 2026
P(0 - N" Ji=1 .A)
log
1==1 (dA))
P(0[A)
P(O 1 N"
1=1
A))((T,
i=1 Noh P(A-= -
1E
P(A= (T, ).
P( N" A)
log
1i=1
d0(d(T.9) (dAA)
P(O
= P(OKT"" N"
A)P(IT;)) NDh P(AA= (7, 1. 1 - P(A = ( )
PIONTTINT A)
log
d0 (0(d N" -1"" (dA),
[
P(ONTT)) " No Ji=1
-"
@
P(ONTT 1M "APTTT
1ill [P((]]]A[AA))) ( A ) P(A-
= 1
log 1d0 (ddd (dA):
= 0,
(13)
where (a) follows the definition of conditional mutual information (Wyner 1978). (b) follows the
fact that the critical tasks and the non-critical tasks are independent (i.e., P(0, (7," N N A)
Ji=l
((O N PAT N" N" yoi
li=1 .A) = P( N" Noi P()T( Ji=1 N
P(A)), and
N" - Ji=1
(c) follows the fact that the non-critical tasks [- are given and thus P([
Ji=1
N" Noh
=
= 0. and thus the information of the non-critical tasks stored the meta- patta--aaemter does not change after the task augmentation.
From (13). we can see that I(:: (7,) N" N K - I(;; (T
(T
Ji=1
K PROOF OF LEMMA 2
In this section, we prove that the learned augmentation P x (A) imposes quadratic regulariza- tion on the original meta-objective. Let's first consider (m:""))), X:). We use Φ; denote the parameter of the task-specific adaptation, i.e., To: (0). Since we use MAML to compute the task-specific adaptation, we know that Φ; = 0 - o... We use 5()) to represent ø to high- light the mixing coefficient A. Note that the action ā indirectly depends on because will affect 3 and ø will affect the distribution of n&. (-15). However, we do not need to directly reason about how A affects because we can capture this relation by analyzing how 8 affects the distribution (-1)) Therefore, we still use the notation ā instead of @(A). The RL objective of the augmented
task T, is:
Ex (m.") Aa)]
(( "0 (. x) log no. (A] (5(A))
Let x; = 1 - A and Fi(x.) log (oB3AA)) Fe (5(A)), ā), therefore, the second-order approx- imation of F.(x;) is:
R(a,)) ~ F.(0) + (0) + (0)=
(14)
We now derive the expression of (0) and (0).
F((x.) (F:(2:) (5(A.)
d5({.) dx:
V.V.ogTT (a]s(A)) A, A (3(A.) + log n.. (a(3(A)) V.ATT Th (3(1). (s' (=1))a)''
+ (( = V. log To. (als) AM (s, a) + log No. (T. (a|s) V.A, [s (6) (15)
26

---

Published as a conference paper at ICLR 2026
We now reason about the second-order derivative:
(xi) = ①
V. To (al A x (B(A.), (OgTa ". (s(A)) -8)
dz.
= -s' S)T V2 log (A:)) 4 (s(A:), (0 "V,A,, (5(A)),
+ log To. (al)()i))
(5(A,), a) (6' - -s),
+ F. = (s' - -s) _ V2 log TØ. (a|s) A (s. + 2(V, To (als) (s.a)
+ log oo. (als) To (s, a) (s' - s).
(16)
By plugging 15 15-- 16 into 14 have that
(z.).) R log no. (als) nes (s,a)
+ log (als)
A Tht (s, a) + log to (als) V.., v.4
(s.a) (s' - -s)[1 2;)
+ (s' 5) V log (als) A,'0(8 (8, + +(V, log (a|s)) V.A," (x,
+ log To. (als) TM (s. 0) (s' s) -1)
log (cls)) (s. a) Cx. (s' =)_
A uor (s, a) "
V2 (s, log To, (als)) (s' - 5] -2i?,
(17)
where (85, = V. log To, (a|s) A,50((s (8,a) + log (als) v.A, (s, (0' 8)
o., Now we take a look at the term V2 log no. ( (aylsz) Recall that the softmax policy parameterization
X₂) (s' - s) T 2(V, log o (als)) V.A,"' (s, a) + lo
(als) (als) 1, (s. (s' s) (1 21
ea7 f()m))
TØ. (als) S" saaoor
therefore we have that
To (als) V2 (o) f(s log Σ ed. No.Td) a'EA V Enes of Van (s Mlsa') + of (Vafls.d))VV ooT f(xaa) d
88 log To,
Ewes e flaaa)
EmeA 00 V Val(s 1o,'')y2
EndEA E0 [(sa0))2
of (s, Emees (s, M(saa)
Evee e0t f(s,a')
Ee oorea oo S( Mas'') (0 Hoa')) EEveA Vaf(s, d' 1(
this
(EvEs e Mo)a0)y²
6. H Hs,a)ao Ev'es 0, V. Viaf(s, f(saa) EvEA c f(a,a))
o V.. (s,aa)
(18)
ell Hoadd) eaI (ind') (Earea )e Menaayy
where H(s,a) a)
0 by Cauchy-Schwartz Schwartz inequality. By plugging (18)
in L7p.
we have that
(x:))) = log T.. (a)s) (s, + (s. + A7 (s) - S)
Φ (s) En'es VT 22 (' A(saa)
o. H(s, a(ss) (s' - s))?
Even [(x,a)) ((e,,a)
log no. (als)A Jo: (8,
Cx, TJ.
27

---

Published as a conference paper at ICLR 2026
( log o. ( () + (x,(s, (0 (f Hx a) (To))
No: (5, Õx, (s, 0- 11 (s.a)0, [
(19)
where (d) follows the fact that Φi = 0 - (TO).
(x,(), = Cx,(o,a) +
A, 1e (8, (s' - T)T oo Ve,l(s,u) - Ed'es of Vanl Kinal)
(s' 8)x7 (x) =
Ed'ed eb (l.u')
(s' 5) > 0 given that ((,aa) > 0. and (x,()))) - (x,
(V J (To)) (s, a) (To))
Therefore, have that
J. (म (-," (0), (r
(e) E (eca))~p te To (a|s)A, (s + (s, - 0 (x
J (0)) + Cr. -0 TT A
where (e) follows 19 Ox = (((,a)=p To, 1Cs, (s, a)}, and lx = E (((a))= o. [1 a)] >
O given that (x, >
O. If we only consider the second-order term, we =o (T (0)) 8T H Therefore, we have that N Ni1 - = L(O, (T," (E 0T (E Nai H where (EET Hx -N
can see that L(O, (A0) Meii (7,t
(6)
0
Ji=1
given that Hx > 0. Thus we have that Ex; P (2) [L (T: (W.) (T," R=1
)] R
L(0, (T, a (> H where A. = ExI (A) [x > 0 given that H > 0.
N
L PROOF OF THEOREM 3
We start with standard uniform deviation bound based on Rademacher complexity Mendelson 2002).
(Bartlett &
Claim 3 (Battlet & Mendelson, 2002)). Let the sample {z1,... , be drawn i.idd from dis- tribution P over and let JF be a function class on Z mapping from Z to a bounded set. Then for 8 > O, with probability at least 1 - 6, it holds that sup 11 ((zi) F s
2R(F, ,',)",,,,)) tion class F.
log(1/8)
where R(F, ... ,2N) is the Rademacher complexity of the func- N
From Claim
we know that the generalization gap 19(Fx)) S R(F,, T.........
log(11/) N
where Fg A (Ji(TT) : Te E Fz). Therefore, we can compute the Rademacher complexity:
N
R(F T T Eo,
Σ (m)))
sup JuF
Nir
< sup
(T))
To
E ET (0.0) o4 [log no, [o (als) (s.
sup
E Fg in P(T) (sa)> oo [(o ((, - log( Σ ebI ((o.))
()AA (s,
sup
a'EAA
where o; is a random variable with equal probability choose and -1. Recall that Φ; = a and 1[V is bounded. Moreover, ward value is bounded, and the chosen feature vector f(s, a) is also bounded. Therefore, there exists [0Th.]
a) is also bounded given that the re-
constant C₁ such that R(F,, T .Tt.) < VN=
sup TO "s.a)>p"" [0T hi] where C.
h h E Therefore, we have that R(F T .TT.) S &C2V No where C2 is pos-
itive constant. Therefore, have that (9(F,)) s 202 A log(1/8)
O
log (1/(1/8)
28

---

Published as a conference paper at ICLR 2026
M EXPERIMENT DETAILS
To update the parameter,,, we need sample 20 trajectories for each task and we use data mixup augmentation to generate another 20 trajectories for the critical tasks. For fair comparison, the baselines use 40 sampled trajectories for the critical tasks to update the meta-parameter.
M. PRACTICAL. CONSIDERATION OF ALGORITHM
In practice, we use warm start to accelerate Algorithm At each iteration k, the upper-level update produces a new mixing-coefficient distribution (A) The lower level samples a set { of mixing coefficients to augment the critical tasks and computes the posterior PM (" lem 6" (f 1
by solving the optimization prob- arg (p) [L(0, (A,,)) (T
in (5) where (0) P* (* (xr,)))). The most straightforward way to solve the optimiza- 5 at each iteration & is that, we randomly initialize ø and solve the optimization problem until convergence get o* (fxr, MMT) However, it requires many gradient-ascen steps which can be computationally expensive. To address this issue, we use warm start. Specifically, in- stead of randomly initializing Φ, we initialize é in iteration k with the parameter (((A,,A-1) learned from the previous iteration & and use one-step gradient ascent to obtain the new parameter o( p({Ar, )= The warm start can significantly reduce the number of gradient-ascent steps because it provides good initialization. Note that the mixing coefficient parameters in two consecutive iterations k- - I and k are close because they are only different in one-step gradient Φx,6 Φ = Agx,A-1 where 3 is a small learning rate and 9A is the gradient. There- fore, it is expected that their corresponding lower-level optimal parameters 6" ({AAs, } and 6² ((), are also close. Note that we only use one-step gradient ascent update & and thus 6((),x11) \ and o² ((\iA-11)))) (N" are different, however,
P (f )
tion problem in
) is updated to- wards 6² ((M.x-1) (NTT) and of (f M.T is initialized from d((Ar,-1) (.) and updated towards ((A ).T) Intuitively, it is expected that (((Ac,)) (TT) will approach o² ((Ar,,)) and fi- nally become o² ((AG,)EE)) when & increases. In fact, has been theoretically guaranteed that x ((Ax,x) will finally reach o" ((As,)) N when increases (Hong et al. 2023).
M.2 FEASIBILITY OF THE AUGMENTED STATE
Recall that the augmented state s = \ + (1 As)s' a convex combination of the two states s and s'. These two states ss, are both valid states because they are visited by the policy. Therefore, their convex combination & will always be feasible state in the state space if the state space is convex. In our experiments, the state spaces are all convex and thus the augmented state 5 is always a feasible state.
M.3 HOW TO FIND POORLY ADAPTED TASKS
The poorly adapted tasks are neither the training tasks nor the testing tasks, but third kind of tasks: validation tasks. Specifically, we have three kind of tasks: training tasks, validation tasks, and testing tasks. The original meta-policy xa is trained on the training tasks, and we find poorly adapted validation tasks. We then identify the critical training tasks to explain these poorly adapted tasks, augment the critical tasks, and retrain meta-policy on the augmented critical tasks and the other training tasks. The generalization is evaluated on the testing tasks. The poorly adapted tasks are automatically selected, and we discuss how to select the poorly adapted tasks in Appendix M.4 M.5] and M.6 Take the MuJoCo experiment and compute the cumulative reward of the meta-policy To after adaptation. We pick the top 20% tasks with the smallest cumulative reward as the poorly adapted tasks.
example,
randomly sample 50 validation tasks
29

---

Published as a conference paper at ICLR 2026
M.4 DRONE NAVIGATION WITH OBSTACLES
We cannot the meta-learning algorithm directly the physical Dr.Dr 2.0 because during training. the drone needs to interact with the environment and can be damaged due to collision with the obstacle and the wall. Therefore, we build a simulator in Gazebo (Figure [ that mimics the physical environment with the scale of 1 1. We run the meta-learning algorithm on the simulated drone in the simulator. By using the simulator, we can avoid damage of the physical drone. Once we obtain learned policy that has good performance in the simulator, we implement the policy on the physical drone and count the successful rate.
Discussion of the sim-to-real problem. the simulator may not have good performance in the real world due to the discrepancy between the simulator and the real world. However, in our case, the sim-to-real issue is not significant because of two reasons: (i) the simulated drone built according to the dynamics of a real Ar. Drone 2.0 (Huang & Sturm 2014); (ii) the states and actions in case are just the coordinates of the location and the heading direction of the drone instead of some low-level control such as the motor's velocity, rotation direction, etc. Given that Vicon can monitor precise pose of the physical drone and the simulator is built on the 1 : scale, if learned trajectory can succeed in the simulator, it can succeed in the real world long as that the low-level control of both the simulated and physical drones can strictly follow the actions.
In some cases, the policy that has good performance in
In this experiment, the state of the drone its 3-D coordinate (xz, y, 2) and the action of the drone is also 3-D coordinate (da, dy, dz) which captures the heading direction of the drone. We fix the length of each step as 0.1 and thus dy
the next state is (xr
+ In this experiment, we do not need the drone
y + (dt
(dd()))
10 (dx)" +(dz)>) (da))
change its height so that we usually fix the value of 2 and set da = O. The goal is 1 square. Denote the coordinate of the center of the goal as (Ignal, Igeut), then for all the different tasks, Tgaol E (0.5, 7.5) and Ugoal E (8, 11)- The obstacle is 3 square. Denote the coordinate of the lower left corner of the obstacle as (Tobstacles yobstacle), then for different tasks,
Tobstaale E (0, 4) and Globstacle 5),
Figure 5: Simulator
M.5 STOCK MARKET
AI for finance (ee et al.) 2025) has been receiving increasing attention. We use the real-world data of 30 constitute stocks in Dow Jones Industrial Average from 2021-01-01 to 2022-01-01. The 30 stocks are respectively: 'AXP', 'AMGN', 'AAPL', 'BA', 'CAT', 'CSCO', 'CVX', 'GS', 'HD', 'HON', 'IMM', 'INTC", 'JNJ', 'KO', 'JPM', 'MCD', 'MMM', 'MRK', 'MSFT' 'NKE', 'PG', 'TRV', UUHH', 'CRM', 'VZ', 'V", 'WBA', "WMT', 'DIS', 'DOW".
The state of the stock market MDP is the perception of the stock market, including the open/close price of each stock, the current asset, and some technical indices (Liu al., 2021). The action has the same dimension as the number of stocks where each dimension represents the amount of buying/selling the corresponding stock. The detailed formulation of the MDP can be found in FinRL (Liu et al. 2021). In addition to the applications to finance, Al has been applied to multiple domains, including graph (Ouyag al. 2024). job shop scheduling (Wang et al. 2025). natural language
(Lan et al. 2025 Kang et al. 2025 Guo et al. 2024). security Guo et al. (2025). and robotics (Qiao
al. 2023)
The RL agent trades stocks on every stock market opening day in order to maximize profit as well as avoid taking risks. The reward function is defined as p1 P2 where Pı is the profit which the money earned from trading stocks subtracting the transaction cost, and pa models the preference of whether willing to take risks. In specific, P2 is positive if the investor buys stocks whose turbulence indices are larger than a certain turbulence threshold, and zero otherwise. The value of p2 depends on the type and amount of the trading stocks. The turbulence index measures the risk of buying a stock (Liu et al. 202I), and lower turbulence threshold means that the RL agent is less willing to take risks. The turbulence thresholds for different RL tasks are different. The turbulence index is a technical index of stock market and is included as dimension of the state (Liu et al. 2021). The turbulence
30

---

Published as a conference paper at ICLR 2026
index measures the price fluctuation of a stock. If the turbulence index is high, the corresponding stock has a high fluctuating price and thus is risky to buy. Therefore, an investor unwilling to take risks has relatively low turbulence threshold. The function p2 is defined as the amount of buying the stocks whose turbulence index is larger than the turbulence threshold. Therefore, the more the target investor buys the stocks whose turbulence index is larger than the turbulence threshold, the larger p2 will be and thus the smaller reward the target investor will receive. For different tasks, we randomly sample the turbulence threshold between 45 and 50.
We first sample 50 training tasks to learn a meta-policy, We then randomly sample 10 tasks and find the top 3 tasks where the meta-policy adapts with the worst performance. These 3 tasks are the poorly adapted tasks. We run our algorithm on the 50 training tasks. To evaluate the generalization performance, we randomly sample 100 test tasks.
M.6 MuJoCo
The target velocity problem of MuJoCo is standard problem for MRL (Finn et al. 2017 Fallah et al. 202T). We follow the standard setting (Finn et al. (017) to study the target velocity prob- lem of two MuJoCo robots: HalfCheetah and Ant, where the episode length is 200. In the original HalfCheetah environment, the reward function is forward reward-cti ctrl cost where the forward re- ward is the velocity of the robot. In the target velocity task, we do not change ctrl ctri and change forward forward_reward reward as -/v Varget where e is the current robot velocity and Vurget is the target velocity. In the original Ant environment, the reward function is fralthy_rewaand -wwoowardrrewaareeee ctrl_cost-contat_.om and we only change forward_rewand as - - -Utagget Following (Finn et al., 2017), we choose the target velocity range of HalfCheetah of Ant as [0, 30. For both HalfCheetah and Ant scenarios, we sample 100 training tasks. We find 10 poorly adapted tasks and run our algorithm on the 100 training tasks to find critical tasks and improve generalization. To evaluate the generalization performance, we randomly sample 200 test tasks.
[0, 2] and the target velocity range
M.7 ABLATION STUDY ON THE AUGMENTATION METHOD
In this section, we include ablation study to show the effectiveness of our augmentation method.
In specific, the previous data mixup augmentation methods (Yao et al., 2021 Wang et al. 2020
Zhang et al., 2018) use predefined Beta distribution P(A) Beta a, a) where = 1 to sam-
ple λ₁- In contrast, we propose to optimize the distribution P(A) by solving the problem (6) To
show the effectiveness of our method, we compare
method that uses the predefined distribution
P(A) = (1, 1) to augment the critical tasks, we refer to this method as "predefined augmenta- tion". We also choose different number of critical tasks Aeti find the case where the predefined augmentation method performs the best. We find that the optimal number of critical tasks of the predefined augmentation method same as the one of our method.
Table 3: Comparison of augmentation methods.
Experiment
Drone
Stock market
HalfCheetah
Ant
Ours
0.97 0.01
421.13 + 12.11
-44.67 + 4.35
119.15 4.02
Predefined augmentation
0.93 + 0.02
394.16 = 16.85
-58.53 + 4.82
109.62 + 5.47
Table 3 shows that our method significantly outperforms the method that uses the predefined distri- bution P(A) = Beta(1, 1) to augment the critical tasks.
M.8 EVALUATION OF THE EXPLANATION
This section evaluates the fidelity and usefulness of the explanation.
Evaluation of fidelity. Fidelity means the correctness of the explanation. Recall that the explanation (i.e.. the critical tasks) aims to identify the most important training tasks to achieve high cumulative reward on the poorly adapted tasks. To evaluate the fidelity, we train a meta-policy on the critical tasks and evaluate the performance of the meta-policy on the poorly adapted tasks. We introduce
31

---

Published as a conference paper at ICLR 2026
two baselines for comparison. The first baseline is the "original meta- meta-policy" that trains all the training tasks. We refer to this baseline as "original". The second baseline is that we randomly pick N training tasks and train meta-policy over the Ne training tasks. We refer this baseline as "random". Note that we chose Ne as the optimal number critical tasks (shown in Figure 3). i.e., 10 for Drone, 15 for Stock Market, and 10 for HalfCheetah and Ant. We compare the performance on the poorly adapted tasks with these two baselines.
Table 4: Fidelity comparison.
Experiment
Drone
Stock market
HalfCheetah
Ant
Ours
0.97 = 0.02
442.29 = 12.79
-37.14 + 5.15
132.62 + 5.15
Original
0.68 + 0.16
296.27 + 35.16
-104.79 12.72
62.47 + 11.03
Random
0.71 + 0.08
284.97 + 29.85
-96.78 + 9.24
65.25 + 3.10
Table 4 shows that our explanation has high fidelity because the meta-policy trained on our expla- nation significantly outperforms the two baselines on the poorly adapted tasks.
Evaluation of usefulness. Usefulness means whether the explanation generalization. Table ! already shows that our method (XMRL) can significantly improve MAML. However, this might be the effect of the task augmentation method. To evaluate whether the critical tasks help improve generalization. We randomly pick Noi training tasks and use the same algorithm (Algorithm I to augment these critical tasks. The choice of Arei same as the one we use to evaluate the fidelity. We refer to this method random, and we compare the generalization of our method with this random method.
indeed help improve
Table 5: Usefulness comparison.
Experiment
Drone
Stock market
HalfCheetah
Ant
MAML
0.87 + 0.01
359.13 + 18.63
-68.89 + 4.36
100.64 + 3.63
Ours
0.97 + 0.01
421.13 12.11
-44.67 + 4.35
119.15 + 4.02
Random
0.89 + 0.02
365.16 11.07
-71.12 5.09
104.98 = 3.65
Table shows that our explanation has high usefulness because randomly pick Nori training tasks and augment can only slightly improve the generalization, while our method can significantly im- prove generalization.
M.9 META- WORLD EXPERIMENT
In this section, we conduct an experiment on ML10 of World. We first validate our observation that "To adapts well to some tasks but poorly to others". In particular, we first use MAML to train meta-policy Ty and evaluate Ty on the test tasks of ML10. We report the results below:
Table 6: MAML generalization on ML10 test tasks (success rate).
door close
drawer open
level pull
shelf place
sweep into
average
0.86
0.35
0.26
0.00
0.00
0.29
Table 6 validates that the imbalanced generalization indeed exists where adapt well to the task of " "door close" but adapts poorly to the tasks of "shelf place" and "sweep into". We next evaluate method MAML+XMRL and the baselines (MAML+TW, MAML+MA, and MAML+MR) ML10. In addition, we compare with two state-of-the the-th--tttttttt MRL algorithms: SDVT (Lee et al. 2023) and ECET (Shala et al. 2025).
32

---

Published as a conference paper at ICLR 2026
Table 7: Performance ML10 test tasks (success rate).
door close
drawer open
level pull
shelf place
sweep into
average
MAML
0.86
0.35
0.26
0.00
0.00
0.29
MAML+XMRS
0.85
0.38
0.25
0.09
0.26
0.37
MAML+TW
0.72
0.29
0.28
0.00
0.24
0.31
MAML+MA
0.82
0,38
0.22
0.00
0.02
0.29
MAML+MR
0.87
0.33
0.26
0.04
0.00
0.30
SDVT
0.08
0,65
0.01
0.00
0.90
0.33
ECET
0.58
0.26
0.24
0.04
0.46
0.32
The results in Table 7 demonstrate that our method can significantly outperform MAML by more than 20% (in terms of average success rate), and outperform the other baselines. Note that while MAML+XMRL outperforms MAML in terms of average success rate, its success rates on "door close" and "level pull"" are slightly lower than MAML. This is reasonable because is not expected that MAML+XMRL can outperform MAML on every task according to no free lunch theorem.
Statistical significance test. We provide p-values from paired t-tests below where choose the significance value a = 0.05.
Table 8: P-values from paired t-tests comparing each method to MAML on ML-10 (a = 0.05).
door close
drawer open
level pull
shelf place
sweep into
average
MAML+XMRL
0.0209
0.0095
0.1192
0.0065
0,0000
0.0081
MAML+TW
0.0001
0.0039
0.1945
0.2589
0.0000
0.2551
MAML+MA
0.0041
0.0183
0.0340
0.4784
0,2850
0.5765
MAML+MR
0,0490
0.3747
0.0159
0.0010
0.4469
0,9293
SDVT
0,0000
0,0000
0,0000
0.7373
0,0000
0.1087
ECET
0.0000
0.0007
0.0021
0.0097
0.0000
0,0586
The results in Table 8 demonstrate that method MAML+XMRL significantly outperforms MAML in terms average success rate as its p-value is under the significance threshold. Among the other five baselines, only the p-vuuue of ECET is close but still above the significance threshold.
Ablation study on the number of critical tasks Neri We vary the number of critical tasks and record the corresponding success rates of MAML+XMRL below. Similarly in Appendix ??, report the performance against the percentage of critical tasks. Please refer to Appendix ?? for design details.
Table 9: Average success rates against percentages of critical tasks ML10
Percentage of critical tasks
0%
10%
20%
30%
40%
50%
Success rate
0.29
0.33
0.37
0.32
0.28
0.20
The results in Table 9 demonstrate that method achieves the highest performance when the critical task are 20% of the training tasks.
Ablation study the augmentation method. To demonstrate the effectiveness of our learned augmentation method, we compare our method to a predefined augmentation distribution P()) Beta(1, 1). Please refer Appendix M.7 design details.
33

---

Published as a conference paper at ICLR 2026
Table 10: Comparison of augmentation methods on ML10.
door close
drawer open
level pull
shelf place
sweep into
average
Our method 0.85
0.38
0.25
0.09
0.26 0.09
0.37
Predefined augmentation
0.83
0.37
0.24
0.00
0.31
The results in Table 10 validate the effectiveness of learned augmentation as it outperforms predefined augmentation distribution.
Evaluation of the explanation. We follow the setup in Appendix M.8 and evaluate the fidelity and usefulness of our method below. Please refer to Appendix M.8 for definitions fidelity, usefulness, and the comparison baselines.
Table 11: Fidelity and usefulness comparison on ML10.
Fidelity Original
Usefulness
Ours
Random
MAML
Ours
Random
0.24
0.00
0.02
0.29
0.37
0.31
The results in Table II demonstrate that our explanation has high fidelity and high usefulness.
N LIMITATIONS
Despite the benefits our proposed algorithms, the limitation of our method is that we require to interact with the environment to collect augmented data, which makes it infeasible for offline RI. cases. We will explore how to extend our method to the offline RL case in future works.
O THE USE OF LLMS
We use LLMs help polish paragraphs in the introduction.
34


Reviews:
[{"review_id": "C21rNIoRhS", "reviewer": "Reviewer_u1Lj", "summary": "This paper presents a method mining, reweighting and augmenting training tasks in meta-RL in order to improve on difficult validation tasks.  Poor-performance tasks are first mined from a large evaluation pool.  Rather than re-incorporate these into the training set directly, this method then weights original training tasks based on their ability to improve performance on the mined set.  Finally, the MRL is fine-tuned jointly with augmentation mixing parameters $\\lambda$ on augmented versions of the critical tasks.\n\nThe augmentation here is a mixup-like linear combination of states drawn from the existing task setup, so that the new augmented states can fill out the convex hull of the original state space.  The state mixing coefficients are optimized to increase mutual information of the distribution over the meta-learned inits theta_0 between augmented and non-augmented tasks, modeled as Gaussians --- that is, it optimizes the mixing distributions to try to get resultant thetas (post-RL-inner-loop) that are different from what what they would have been training on just the original tasks, making the augmentation actually provide different datapoints for the outer MRL step.\n\nTheoretical convergence results indicate soundness of the approach.  The method is evaluated on multiple MRL benchmarks and settings, showing substantial improvement in poorly performing tasks with little regression in those that already perform well.", "questions": "* It might also be interesting contrast the selection of critical tasks to coreset selection\n\n* Alg. 1:  This doesn't say clearly when (or whether or how much) to run the MRL learning/tuning to update $\\theta$ and how many times this should be run to estimate the P(\\theta) distribs, all of this action seems to be subsumed in the last sentence \"Estimate P*.\"\n\n* l.145:  \"We include the algorithm to solve the problem (2) in Appendix C.\" --- Eq 2 looks like it would requires rerunning the the entire meta-learning loops to solve the inner argmax_theta:  this argmax is exactly a weighted form of eq (1).  however the algorithm in Appendix C doesn't seem to do that, what is actually done here is more of an alternating coordinate descent.  What is actually done should be explained more up-front in the main text.\n\n* line 326 says P(theta|{T^cri}) (non-augmented; no bars) is a marginalization of P(theta|{Tbar^cri}) (augmented with bars) over all sampled mixture coeffs lambda.   How is this possible,\ngiven that the non-augmented tasks are the source of the states forming the convex hull of hte augmented states?  Shouldn't they be evaluated at lambda=0 or 1 to get the T^cri samples?  Or does this distribution mean something else?\n\n* It looks likely that critical task weights will tend to be lower for well-represented tasks and higher for underrepresented (though this is not the only factor).  In a limiting case, if there are duplicate tasks in the meta-training set, the weights can be spread among them.  So if tasks that are near-dups are the ones important for a poor performing task, the weights may be lower and they could be missed in the mining selection.  Does this happen or is it not an issue in practice?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The approach of optimizing augmentation parameters to explicitly result in different theta points (not just different mixing states) is quite interesting, and evaluations compared to predefined mixing demonstrate its effectiveness over a reasonable uniform baseline.  The task mining step is also formulated well and makes a lot of sense.", "weaknesses": "While there are some good results compared to appropriate baselines and good ideas (especially the mutual information for augmentations), I had a lot of trouble reading and understanding the paper and method.  (Note, I'm not an expert in RL, but do have familiarity with the subject).\n\nMany of the objective formulations are presented at a high mathematical level while relegating crucial explanations to the appendices.  This is especially true of the core equations 4, 5 and 6, along with Alg 1, which offer definitions of objectives but aren't very clear on the actual steps performed by the algorithm in updating them.  There is also a lot to keep track of here, and simpler explanations and/or diagrams would help.  As I think the ideas are good and have promising results, I'm confident the explanations can be simplified more.\n\nAppendix M is another case in point:  These are all good experiments and explanations illustrating the behavior of the method and its results quite well, and also provide concrete examples of the tasks and how the MRL state interpolations fit together.  Putting more of this in the main text would help ground the explanations and highlight key results like performance improvements on the poor tasks used in the mining step.  (Though I recognize this may be more my opinion, pushing these aside in favor of the convergence results in the main text is exactly backwards --- while the convergence results are good to see and summarize, I think much of Appendix M offers more explanatory value of the core ideas in the method).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Xn1ahB1ynI", "reviewer": "Reviewer_4uvj", "summary": "This paper addresses generalization in meta-reinforcement learning (meta-RL) by introducing a novel two-stage approach that leverages example-based explanation and information-theoretic task augmentation.Stage 1 identifies critical training tasks most relevant to poorly adapted tasks through a bilevel optimization that assigns task-specific importance weights. Stage 2 improves generalization by maximizing conditional mutual information (CMI) between the meta-policy and augmented critical tasks. The authors use a learnable task mixup augmentation strategy to achieve this, rather than fixed data-mixing rules. Theoretical analysis guarantees O(1/\\sqrt{K}) convergence and improved generalization bounds. Experiments on real-world (drone, stock trading) and simulation benchmarks (MuJoCo, Meta-World) show consistent gains over MAML and recent meta-RL improvement baselines (task weighting, meta-augmentation, meta-regularization).", "questions": "1. In Eq. (2), when N^{tr} becomes large, does the bilevel optimization become unstable or computationally infeasible? How is this mitigated in practice?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. New conceptual link between explainability and meta-RL: the example-based explanation mechanism to identify “critical” tasks is creative and intuitively appealing.\n2. Information-theoretic formulation: Using conditional mutual information to formalize “attention” toward critical tasks provides a principled grounding and unifies ideas from explanation, data augmentation, and meta-learning.\n3. Theoretical guarantees: The paper presents convergence and generalization proofs, which, while incremental, give some rigor to the framework.\n4. Empirical validation: Comprehensive experiments across four environments show clear improvements over baselines, with meaningful ablations (number of critical tasks, learned vs. fixed mixup distribution).", "weaknesses": "1. Meta-RL setup realism:\nThe method inherits the common limitation of meta-RL—assuming access to an oracle task distribution P(T) from which both training and test tasks are sampled. In real applications, such a distribution rarely exists, and constructing it requires a near-perfect world model. Therefore, while the algorithm is technically solid, its practical significance is limited.\n\n2. Limited novelty in theory: 1) Theorem 1 (convergence) essentially restates standard SGD-style results for bilevel optimization [1]; 2)Theorem 3 (generalization bound) reduces to the observation that increasing the number of training tasks improves generalization, which is unsurprising and  closely parallels Theorem 1 in (Yao et al., 2021) [2].\n\n3. Conceptual gap between optimization levels: In Eq. (2), the upper-level objective optimizes the weights \\omega to maximize returns on poorly adapted tasks, whereas the final meta-RL evaluation concerns adapted tasks. The link between these two objectives could be clarified, as there may exist a performance gap.\n\t\n4. Scalability concerns: The approach requires solving multiple nested bilevel problems (for both task weighting and CMI maximization). When N^{tr} is large, this may cause optimization difficulties or high computational cost.\n\t\n5. Mutual information interpretation: While MI provides a neat information-theoretic lens, it is not fully clear how optimizing MI translates into concrete improvements beyond encouraging task diversity. The relation between the learned MI term and mixup augmentation could be elaborated further.\n\n\n\n[1] Jun Shu et al., Meta-Weight-Net, NeurIPS 2019.\n\n[2] Improving Generalization in Meta-learning via Task Augmentation, ICML 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W5wPBWHcnL", "reviewer": "Reviewer_HExQ", "summary": "This paper addresses the problem of imbalanced generalization in Meta-Reinforcement Learning (MRL), where the meta-policy θ exhibits performance disparities when adapting to new tasks. To tackle this issue, the paper proposes a post-hoc improvement method. First, it identifies \"critical tasks\" that are most beneficial for improving performance on \"poorly-performing tasks\" through a bilevel optimization problem. Subsequently, the paper formulates another bilevel optimization framework to learn the optimal data augmentation for these critical tasks. The upper-level objective maximizes the Conditional Mutual Information (CMI) to ensure that the augmentation provides maximum additional information. The lower-level objective updates the meta-policy distribution based on the current augmentation strategy. Through this process, the algorithm iteratively optimizes the data augmentation strategy (specifically, the sampling distribution for the mixup coefficient λ). The authors theoretically prove the convergence and generalization improvement of their algorithm and validate its superior performance through experiments on MuJoCo, Meta-World, and real-world tasks.", "questions": "see weakness", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The research direction of this paper holds significant industrial value and practical relevance. In industrial applications, there is a strong emphasis on synchronous and balanced convergence across various tasks, with a particular focus on the performance on long-tail or difficult tasks. Identifying critical training tasks is indeed key to enhancing performance on downstream poor-performing tasks from my point of view.\n\nThe method of identifying critical tasks by finding an optimal weight vector w is clever and intuitive, with a well-formulated optimization objective that aligns with cognitive reasoning.\n\nThe introduction of the CMI concept to construct the optimization objective for learning the sampling distribution of the mixup coefficient λ is reasonable and promising. Furthermore, the use of an augmented dataset {\\hat{T}_{cri}} to estimate the posterior P(θ|{T_{cri}}) via expectation (which can be viewed as an application of the total probability formula) is an approximation method.\n\nThe mathematical derivations in the paper are robust, with a solid formulation of the problem and a well-established convergence proof.", "weaknesses": "As mentioned in the paper, this data augmentation method requires online MDP tuples from interaction with the environment, which seems to be unavailable in offline RL settings for now.\n\nThe overall method appears complex. It first requires finding the weights w and then proceeds to optimize the mixup sampling distribution parameters for λ, leading to high computational complexity.\n\nThe approach of using poorly-performing new tasks to supplement the training data (via data augmentation) might be viewed as \"hacking\" the test set. It does not seem to enhance the real generalization capability of MRL from a more universal perspective.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "y0EU8nT4oe", "reviewer": "Reviewer_kbyg", "summary": "This paper primarily aims to improve overall generalization performance by identifying critical tasks and further strengthening them through data augmentation. \n\nRegarding the identification of critical tasks, the authors define the algorithm as finding the optimal weights of critical task pairs. I checked the proofs in the appendix, and it seems to be correct. \n\nAs for further improving generalization performance by augmenting critical tasks, the authors propose optimizing the previously predefined distribution P(λ) to maximally store information. This is also formulated as a bi-level optimization problem, for which the authors provide a tighter generalization bound. \n\nExperiments are conducted both in MuJoCo simulations and on real robotic systems.", "questions": "See weakness above", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. I appreciate the authors can state the empirical observation on performance degradation, which I find is significant to make the paper complete.\n2. I also appreciate the authors could hold an ablation study to compare with the policy obtained through (2) to further validate the role of task augmentation.\n3. The experiments are conducted even on real-robot, which is very impressive for me.", "weaknesses": "1. After reviewing the theoretical derivation, I feel it might be problematic. \n\nIn Appendix E DERIVATION OF THE CONDITIONAL MUTUAL INFORMATION: line 819, where is the distribution P(\\overline{\\mathcal{T}}). Line 827, why the integral concerning P(\\overline{\\mathcal{T}}) is removed.\n\nIn Appendix F line 859, why extending the distribution of P(\\overline{\\mathcal{T}}) can be transformed to P(\\lambda_i). In line 870, I do think the fact that P(\\overline{\\mathcal{T}_{i=1:N}} | \\mathcal{T}_{i=1:N})  = P(\\lambda_i) is wrong. The left-hand side is a joint distribution over N augmented tasks (given the original tasks), while the right-hand side is merely P(\\lambda_i), which is not only ambiguous in terms of the index I, but also represents only a single-variable distribution, not a joint distribution over N variables. Furthermore, even if we only consider one-task condition, I do think P(\\overline{\\mathcal{T_i}} | \\mathcal{T_i}) not equals to P(\\lambda_i). Mathematically, it should be \\int P(\\overline{\\mathcal{T_i}} |  \\mathcal{T_i}, \\lambda_i) P(\\lambda_i) d\\lambda_i\n\nI feel that these issues have seriously compromised the theoretical rigor of the paper, so I did not proceed to check the subsequent theoretical content further.\n\n2. The authors propose to augment the state by convex combination, but I do think in many scenarios, the convex combination of state would not be acceptable. Also, I think this is a strong assumption for applying the algorithm. Therefore, the authors at least should make this clarified.\n\n3. In meta RL, while most online methods emerged around 2021-2022, several offline methods also aim to improve generalization ability, including those also based on information theory [1]. I believe discussing these works properly would be beneficial for providing a more complete literature review.\n\n4. Could you please give me an intuition about why only minor degradation on very few tasks happens when focusing more on critical tasks? Maybe this is a combinatorial phenomena stems from  the number of critical tasks and the optimized distribution P(\\lambda) ?\n\n5. Solving the proposed algorithm needs to optimize two bi-level problems iteratively. It seems that there may exist some instability. Do the authors use some tricks to stabilize the training process?\n\n[1] Towards an information theoretic framework of context-based offline meta-reinforcement learning", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)