Evaluated CMI Bounds for Meta Learning: Tightness and Expressiveness
Authors: Fredrik Hellström
Giuseppe Durisi
Venue: NeurIPS 2022 Accept (2022)
Keywords: Meta learning
information theory
generalization bounds
PAC-Bayes

Abstract: Recent work has established that the conditional mutual information (CMI) framework of Steinke and Zakynthinou (2020) is expressive enough to capture generalization guarantees in terms of algorithmic stability, VC dimension, and related complexity measures for conventional learning (Harutyunyan et al., 2021, Haghifam et al., 2021). Hence, it provides a unified method for establishing generalization bounds. In meta learning, there has so far been a divide between information-theoretic results and results from classical learning theory. In this work, we take a first step toward bridging this divide. Specifically, we present novel generalization bounds for meta learning in terms of the evaluated CMI (e-CMI). To demonstrate the expressiveness of the e-CMI framework, we apply our bounds to a representation learning setting, with $n$ samples from $\hat n$ tasks parameterized by functions of the form $f_i \circ h$. Here, each $f_i \in \mathcal F$ is a task-specific function, and $h \in \mathcal H$ is the shared representation. For this setup, we show that the e-CMI framework yields a bound that scales as $\sqrt{ \mathcal C(\mathcal H)/(n\hat n) + \mathcal C(\mathcal F)/n} $, where $\mathcal C(\cdot)$ denotes a complexity measure of the hypothesis class. This scaling behavior coincides with the one reported in Tripuraneni et al. (2020) using Gaussian complexity.

Evaluated CMI Bounds for Meta Learning:
Tightness and Expressiveness
Fredrik Hellström
Giuseppe Durisi
Chalmers University Technology
Chalmers University of Technology Gothenburg, Sweden
Gothenburg. Sweden frehells@chalmers.co
Abstract
Recent work has established that the conditional mutual information (CMI) frame- work of Steinke and Zakynthinou (2020) is expressive enough to capture general- ization guarantees in terms of algorithmic stability, VC dimension, and related com- plexity measures for conventional learning (Harutyunyan et al., 2021, Haghifam al., 2021). Hence, provides a unified method for establishing generalization bounds. In meta learning, there has far been divide between information- theoretic results and results from classical learning theory. In this work, we take a first step toward bridging this divide. Specifically, we present novel generalization bounds for meta learning in terms of the evaluated CMI (e-CMI). To demonstrate the expressiveness of the e e-CMI framework, we apply our bounds to representa- tion learning setting, with n samples from tasks parameterized by functions the form f. h. Here, each f. F is a task-specific function, and h E is the shared representation. For this setup, show that the e-CMI framework yields C(F)/n, where C(-) denotes complexity
bound that scales as /C(H))(mm)) measure of the hypothesis class. This scaling behavior coincides with the one reported in Tripuraneni et al. (2020) using Gaussian complexity.
1 Introduction
Meta learning, sometimes referred to as learning to learn, is process by which performance on a new machine learning task is increased by using knowledge acquired from separate, but related, tasks [1, 2]. Concretely, the meta learner Á has access to training data from several different tasks, which are embedded in a common task environment, and aims to extract information from this data. task
The goal is from the same task environment. For instance, the task environment can consist of different image classification tasks, and the goal of the meta learner is to learn a shared representation for the tasks or to find suitable hyperparameters for a base learner performing image classification.
this information to improve the performance of base learner A on
As in conventional learning, a central goal in meta learning is to bound the gap between the loss on the training data and the population risk on unseen data. Two current approaches for achieving this goal are: i) use techniques from classical learning theory to obtain minimax performance guarantees, or ii) to information-theorrt guarantees. So far, these two lines of work have evolved largely separately. In this paper, we take some steps toward unifying them. Specifically, we: i) derive new, tighter information-theoree generalization bounds for meta learning, and ii) demonstrate that these bounds are expressive enough to recover bounds for meta learning from classical learning theory. To concretize the discussion in this introduction, we assume that the meta learner outputs member h of function class H. on the
methods to obtain algorithm-, data- and distribution-dependee
36th Conference on Neural Information Processing Systems (NeurIPS 2022).

---

basis of n samples from á different tasks, and that base learner selects a member f of a function class F, on the basis of the output of the meta learner and n samples from a given task.
Classical learning theory for meta learning. The theoretical analysis of the benefits of meta learning in terms of loss bounds dates back to [3], where the notion of task environment was formally introduced. More recently, for the setting of representation learning, [4, Thm. 5] derived risk bound that scales as' VC(H)/n + vC(F)/n, where C(.) denotes a complexity measure of the function class. This demonstrates the benefit of meta learning for tasks that share a common environment. Indeed, in the conventional single-task learning scenario, the samples from a given task need to be used for learning h and f simultancouslyy leading to a /C(H. x F)/n bound. The bound provided in [4, Thm. 5] was later improved by [5] to a scaling of VC(H) scaling, where C(H) decays with the product nú, confirms the intuition that all of the nú samples that are observed are informative at the environment level. Meta learning has also been extensively studied in several special cases. For instance, [6, 7, 8] study setting with linear features and task mappings, while [9, 10] consider online convex optimization setting. In this paper, will mainly focus on the representation learning setting.
C(F)nn. This improved
Information-thhereee generalization bounds. For conventional learning, the study of information- theoretic bounds was initiated by [11, 12), where the average generalization gap of a learning algorithm is bounded in terms of the information that the algorithm reveals about the training data. At its heart, this line of work relies on a change of measure technique that relates the training loss to the population loss. While the first iiformation-theoreeee bounds were given in terms of the mutual information between the output of the learning algorithm and the full training data, recent works provide bounds in terms of the disintegrated mutual information between the loss that the algorithm incurs on a single sample pair and a selection variable indicating which sample is used for training, given a supersample containing both the training and test data. These developments are due to the samplewise approach of [13], the disintegration introduced in [14], the evaluated conditional mutual information (c-CMI) notion from [15], and combinations and extensions of these from [16, 17, 18, 19, 20]. This line of work is also intimately related to PAC-Bayesian generalization bounds [21, 22], where the generalization gap, averaged over the learning algorithm, is bounded with high probability over the data in terms of KL divergence. This is explored further in [23, 24).
Information- analysis of meta learning. Recently, information-theoretti generalization bounds have also been applied to meta learning [25, 26, 271. In parallel, a PAC-Bayesian analysis of meta learning has also been developed [28, 29, 30, 31, 32, 33). Generalization bounds obtained via information -theorettcc methods have also been used as training objectives in order to improve performance [30, 34). The quantity of interest in meta learning is the meta-population loss, which is the population loss evaluated on a task that was not observed during the meta learning phase. While this quantity is unknown, the meta learner has indirect information about it through the observed meta-training loss, which is the loss that the meta learner incurs on the training samples from each of the observed tasks during the meta learning phase. The standard approach in the ceemmatiio-thhorettt and PAC-Bayesian analysis of meta learning consists of two steps. The first step involves bounding the difference between the meta-training loss and a suitably defined auxiliary loss. The second step involves bounding the difference between the meta-population loss and the auxiliary loss. The two natural candidates for this auxiliary loss are the population loss of an observed task and the training loss for an unobserved task. One of these steps (the first or second, depending on the choice of the auxiliary loss) is purely at the task level, while the other is purely at the environment level. This makes it possible to view each of these steps a conventional learning problem. so that a standard information-thhoreee
generalization bound can be applied for each step. By the use of the triangle inequality, the two bounds are then combined to obtain a bound on the meta- populatool loss in terms of the meta-training loss. We will refer to this procedure as a two-step derivation. An alternative approach recently used by [26], where one-step procedure was employed. Rather than relying on an auxiliary loss, [26] immediately bounds the difference between the meta- potttapppllation loss and the meta-training loss in terms of a mutual information that captures both task level and environment level
In the interest of brevity, we suppress logarithmic factors throughout this section.

---

dependencies. The environment and task level dependencies can then be obtained by decomposing this mutual information. The resulting bound turns out to have a better scaling with á than the two-step bounds. However, the nntrmattion-thorreeee analyses of meta learning reviewed so far do not provide any rigorous characterization of the scaling behavior of the bounds. In particular, the dependence of the information measures on the sample size is typically ignored. This precludes a direct comparison between these information -thhrrrtt bounds and classical learning theory results.
Contributions. Focusing on the meta learning setup, we present novel information- -hthorttttt bounds based on the e-CMI framework and demonstrate how to recover minimax results from classical learning theory via these bounds. Our specific contributions are as follows. In Section 3.1, we derive bounds for the average generalization error in terms of the disintegrated, samplewise e-CMI of the meta learner and base learner: in Theorem 1 and 2, we provide square-root bounds, which are shown to be tighter than results in the literature; in Theorem 3, we derive novel bounds in terms of the binary KL divergence. For low values of the training loss, the binary KL bounds display a more favorable dependence on the number of data samples than the square-root bounds. Next, in Section 3.2, we extend these average bounds to obtain gigh-probabilityy generalization guarantees. This is necessary to perform comparisons with -robbprbyybylityy bounds from classical learning theory. Finally, in Section 4, we demonstrate the expressiveness of our bounds by applying them to a representation learning setting. Under certain assumptions about the hypothesis classes, we provide upper bounds on the information measures that appear in our bound in terms of complexity measures. The results that we obtain via this procedure display a scaling behavior that coincides with the one reported [5]. This demonstrates that the e-CMI framework is expressive enough to recover the scaling behavior of generalization guarantees for meta learning obtained via classical learning theory.
2 Problem Setup and Notation
We now introduce the meta learning setup that we consider throughout the paper, as well as the necessary notation for stating our results. Similar to [3], we consider task environment formulation that includes the representation learning setting of [4, 5] as special case.
Our meta learning setup involves the following quantities. We consider a task distribution D on the task space T. For given task T, there is corresponding in-task distribution D, on the sample space Z. The goal of the meta learner is to output a meta hypothesis U E U. This is done on the basis of n samples from ń tasks. Formally, the meta learner is a mapping Á gnx x R. + U, where the random variable Å E R captures the potential stochasticity of the learner. The goal of the base learner is to output a hypothesis W E W, given the output of the meta learner and n samples from a specific task. Formally, the base learner is a mapping A 2 x R x l + W. The random vector R- (R1, (g)) E R has entries that capture the potential stochasticity of each base learner. The entries are independent from the data and assumed to be identically distributed. Here, the spaces V and W may be function spaces or parameter spaces, depending on the learning algorithms.
Within each task, the training set for the base learner is randomly formed from a supersample according to the conditional mutual information (CMI) framework of [15]. Specifically, for given lask r, let 2 E n denote the supersample, which is an n x 2 matrix with elements generated independently from D,. For convenience, we index the two columns of Z by and and the rows by 1,..., n. The training set Z3. is formed on the basis of membership vector S 5= (S), ... S.), with entries generated independently from Bern(1 (1 /2) distribution. More precisely, the jihh element of Z5 is given by [zg]; = Zj.ss, i.e., the S,th element from the jth row of Z Furthermore, we let - (1 Si,..., 1 S.,) denote the modulo-2 complement of S, which we use to form the test set Z_g. whose jth element is given by [ = -s With this construction, we randomly assign each sample in the supersample to either the training set or test set with equal probability.
We now describe the mete-supers Z, which contains 2n samples from 2n tasks, as in the meta-learning extension of the CMI framework provided in [35]. Throughout, we let i {l,.... f.} denote a task index, j E {l,..., n denote a sample index, and k, E {0, 1} denote binary indices
While identical distributions are necessary for our results, this assumption simplifies the presentation.
3

---

In-task
supersample Z
Sample
pair Z 1
1.0
1.0
1,1
1,1
2
2
Task membership
2,0
Z
2,1
2
2,0
2,1
2,0
2,1
Task index
2 =
z1.0
z1.0
3,0
3,1
3,0
3,1
3,0
3,1
3,0
3,1
Sample index
Sample memhership
Z
Z4,1
2
22.1
Z
2
4,0
4,1
4,0
Z
1.51 St
Task pair Z
51.8,
Z
22.0
S
1
1
0
71.
22.0
2,1
0
0
1
z5
$
S
2,00
1
3.0
3,0
Z
7
Meta membership
1
1
4.1
Sample membership
Meta-training set
Figure 1: A graphical representation of our notation. this example, the meta-supersample contains two task pairs: Zt, which is marked in blue, and Z In turn, Zt consists of the two in-task supersamples z².0 and z which is marked in brown. Next, 2 consists of the four sample pairs 2 where j = :.....4. the figure, Z is marked in purple. Finally, Z consists of pair of samples. Z and Z When binary vector is used subscript or superscript, this indicates that we should enumerate the meta meta-superssmmle according to that vector. To illustrate this,
consider the construction of the meta- meta-traing training set z The meta- training set is formed on the basis of the meta-subset choice S and the observed-task subset choice S For instance, from the first task-pair, S1 indicates that we should select task 1. Then, from the first sample pair in this task, S1, indicates that we should select sample 0, which Z marked green. Repeating this for each sample pair in the in-task supersample z1,1, we can identify the remaining elements of Z S This procedure is performed for each task index until all samples of z are identified. The meta-test set ~ : is formed by an analogous procedure, but is now based on - and -S- The entries from the mute-supeee that selected for z_5 are marked in orange.
indicating task membership and sample membership respectively. Formally, the mupersuuee Z can be viewed as a data structure with 4n clements. In Figure 1, we illustrate Z as an x 4A matrix for the case of î = task pairs and n = 4 sample pairs for each task. We decompose Z as ( 2"), where each element can be seen as a task pair. Specifically, the pair Z can be decomposed as (Z zz-2), where each element a task-specific supersample as described above. The task-specific supersamples are Zik (Z
Zik)), where each element is pair of data samples. Specifically, each sample pair is Zj = (Zj,0 i.k Ziih), where Z E Z. The elements of are generated as follows. First, we generate T4,6 D. Then, we independently generate the samples Z Dr... This is repeated for all indices to form Z.
j.0'
Finally, we describe how the meta-training data selected from the meta-supersample Z. This done on the basis of the meta-membership vector Š and the in-task membership vector Specifically, the meta membership vector Š ($ ........... Sa) is an ñi-dimensional vector, while the sample membership vector S (S1.0, g1.1 .e....... Sñ.0 Sa.1) is a collection of 2n vectors, where ...., generated independently from Bern(1/2 (1/2) distribution. For any Bernoulli matrix X, we let - denote its elementwise complement modulo 2, i.c., X, where 1 is the all-one matrix.
each Si. = (St ...., k),A) is an n--imensional vector. The elements of all these vectors are

---

These membership vectors are used to form the meta- veeta-taiinng set as follows. We use the convention that, when a binary vector is used as a subscript or superscript of Z, this indicates that we should enumerate over this vector. Using this convention, the training set for the (i. k)th task is constructed as Z.ght. = ( 1,51 1. Zid w..hk
We will use the shorthands Z g and Z = Zi,k 1,5)
The test set for the (i, k)th task zid. is constructed analogously, but on the basis of S k_ The full
meta-training set is z = (Z _t, ...... 25.3, ). and the meta-test set ZーŠ is defined analogously. With this construction, each täsk in the meta- supersspper is assigned to either the meta- training set or the meta-test set with equal probability. Then, as before, the samples within each task are assigned to an in-task training set test set with equal probability. The meta-training set consists of training samples within training tasks, while the meta-test set consists of test samples within test tasks.
We denote the output of the meta learner as U A(zğ, R) E u and the output of the base learner for task ki) as Wik A(Z Rí, U) W. The performance of the learners is evaluated through a loss function e W x 2 + [0. IJ. We denote the losses that the meta learner and base learner induce on the meta-supersample Z by X, which inherits the subscript and superscript notation that we described for Z. Thus, we have x Agd f(Wik, 23,1)
z.k)). In other words, X,1 is the loss induced on
the (j, 1'th sample in the (i, k)th task.
On the basis of the loss matrix À and the membership vectors á and S, we can compute four different losses. The main quantity that we are interested in bounding is the average meta peeta-populttoon loss LD. which is the loss on test data for unobserved tasks. The quantity that the meta learner has access to the average meta- meta-training loss, Σ, which is the training loss for observed tasks. The other two losses are the average auxiliary test loss, L. which is the loss on test data for observed tasks, and the average auxiliary training loss L, which is the loss on training data for unobserved tasks. In the two-step derivations, one of these two quantities is used the auxiliary loss. These four losses are given by
A,
hn
Lo =
Ex
9:
i
S.
nn JJ=1
55
nn
1183
rJ=1
A,D
6,n
L
Exg 2
8.
(2)
TH J
3.93
1-=1
Finally, we end this section by introducing some information-theoreii quantities that appear in our bounds. First, let P and Q be two probability measures such that P is absolutely continuous with respect to Q The KL divergence between P and Q is denoted by D(P []Q)). For the special case where P and Q are Bernoulli distributions with parameters p and g, we let
d(p = D(P Q)) plog
!
1 + (1 p) log
(3)
We refer to d(p 9) as the binary KL divergence. The mutual information between the random variables X and Y is given by T(X;Y) = D(Pxy || Px Pr). where Pxy is the joint distribution of and Y and Px and Py are the corresponding marginals. The disintegrated mutual information between X and Y given third random variable Z is given by (xx = D(Pxyizll PxizPriz). where PxYi2 is the conditional joint distribution of X and Y given Z and PxizPyye is the product distribution formed from the corresponding marginals. The expectation over Z of the disintegrated mutual information is the conditional mutual information T T(X; Y))) = Ez [ 1 ())
3 Generalization Bounds for Meta Learning with e-CMI
In this section, we present generalization bounds in terms of the e-CMI of the meta learner and base learner. In Section 3.1, we derive average square-root bounds that tighten results from [26, 35), as well as novel binary KL bounds. In Section 3.2, we extend these results to obtain novel, high-probability information-theomm bounds for meta learning. In Section 4, we demonstrate the expressiveness of
5

---

the e-CMI framework by using the bounds from this section to recover generalization guarantees from classical learning theory for representation learning.
3.1 Average Bounds
In Theorem 1, we present a square-root bound the average generalization error obtained through two-step derivation. Specifically, one step consists of bounding the unobserved training loss L in terms of the observed training loss L, and the second step bounds the meta-popuus totta-ppplation loss LD in terms of L. Chaining these two bounds, we obtain bound on Lp in terms of L. The bound depends on the information captured by two random variables: the task-level variable X which contains the training loss and test loss for task (i, S.), well as the environment-level variable S3ssy. which contains the training losses for both the observed task (i, S;) and the unobserved task (i, St).. We provide the proof of this result in Appendix A, along with the proofs of all other results in this paper.
Theorem (Two-step square-root bound). Consider the setup described in Section 2. Then,
LD
R2 2,91
22 (X (.) +
E 2,8
212.8.
(2g S, 55 (4)
nn
nn
The first term captures the environment-leww generalization error while the second term captures the task-level generalization error. In order clarify the relation between Theorem and results from the literature, we relax it by upper- hopur-bounniig the disintegrated individual-sampae e-CMI terms by their integrated, full- sample, parametric CMI counterparts.
Corollary 1. Theorem implies that
LD - 1 21(U;
21 9, 9 S1,
12 $1)
(5)
This recovers the result of [35, Thm. 1]. demonstrating that Theorem is tighter.
Next, we present alternative square-root bound that is obtained through one-step derivation. This bound depends on the information captured by the random variable AJ. which contains the training and test loss for both the observed and unobserved tasks.
Theorem (One-step square-root bound). Consider the setup described in Section Then,
LD T.
Ez
(2.
nn
Again, to compare this bound to results in the literature, we relax it by upper- pound-bounding the disinte- grated individual-sune -smple e-CMI terms by their integrated, full-sample, parametric counterparts.
Corollary 2. Let W (W1.9, w and W {W*1A Ti=r Then,
21(W; 21(W;S,512 S,SIZ)
(1(U$,,512 251
L.D 7
nn
21(U; ( 2hI(W) Z
(8)
n
Up to some constant factors, this recovers the result in [26. Thm. 5.1]. Note that, if Á or A are deterministic learning algorithms with continuous outputs, the mutual information terms in (8) are unbounded. In contrast, the CMI terms in (7) are always finite. This is discussed in more detail in [15]. Furthermore, the bound in (7) compares favorably to [35, Thm. 1], since decays with the product zó rather than with n and à separately. This improvement is due to the one-step derivation.

---

Finally, in Theorem 3, we present two novel bounds in terms of the binary KL divergence. The advantage of these bounds, as compared to the square-root bounds Theorem I and 2, is that they have more favorable dependence on the number of samples for low training losses. We demonstrate this improved rate for representation learning in Section 4.
Theorem (Binary KL bounds). For m 2 2, q E [0, 1] and c > 0, let
dm' sup p p (0, q + S c
(9)
Then,
R,n L
a,n
LD S d2
tz'
Σ
10 J.5j
S,12, 8T)
Σ 1
S 1025-51
(10)
ni
i,
Furthermore,
LD + L + L
S di'
L T nu
Σ
(11)
Interestingly, (11) provides a bound on the sum of the average meta veta-populattin loss Lp. the test loss on observed tasks L, and the training loss on unobserved tasks L. Due to the nonnegativity of the loss, obtain explicit bound on Lp by using the lower bound L + Σ 0, which is sensible relaxation when Lp is the dominant term. By this relaxation, we weaken the bound at most by a constant factor. As previously mentioned, the bounds Theorem 3 can have a more favorable dependence the number of samples than the square-root bounds in Theorem and when the training loss is low. the following corollary, we present a bound on LD for the case where 7,
Corollary 3. Assume that L == 0. Then, Theorem 3 implies that
n.n
n.a
4
LD S 4 4exp
S,,
(12)
n
Compared to the bound in Theorem 2, there is square root in Corollary 3. As we show in Section 4, this can lead to faster rate of decay with the number of samples.
3.2 High-probability Bounds
In the previous section, provided bounds the average generalization error. However, meta learning bounds obtained via classical learning theory are typically high-probability bounds [4, 5J. In order to assess the expressiveness of the e-CMI framework in terms of its ability to recover these results, we now extend the bounds from Section 3.1 to the grobbbbbbbyllty setting. For this, we need some additional notation. We let Lp (2, 5, S) and Ż(Z, S, S) denote the meta- popellooptttionn loss and training loss given that the meta-training set is constructed from (Z, S)S. Specifically,
LD(Z,
1
BR.. [(A(Z))- R, A(Z R)) R)) -
(13)
nu +J=1
L(Z,
ER (AA(Z)S) R. A(23, R)), 2,55
(14)
TIñi
We now present high-probabilityy version of the two-step square root bound in Theorem 1. To simplify the presentation, we omit explicit constants and assume that Á and AA are indifferent to the order of the data samples. The theorem statement is provided in more general form in Appendix A.

---

Theorem 4 (High-probabiln probability two-step square-root bound). Let Q1.s, denote the conditional distri- bution of 2 given (Z, S, S), and Pi.s, denote Q1,,), Furthermore, Q!.-ši denote the conditional distribution of X2- given (Z, S2, and let p2.- denote gg Q2.-51 Then, there exist constants C1 C2 such that, with probubility at least 1 1 under the draw of (2, S,)
D(Q1,s, || Pi,s) + log( nya
S sCi
à
2 D(QI. 81 pl, + log( 5
+ C2
(15)
n
The KI. divergences in (15) can be interpreted as pointwise e-CMIs. Indeed,
((),,,, (1S, 2,5.5 D(Q | -(x s1 Sz)
Ez,,, ( Pi.s,)
(16)
(17)
Finally, we present a high- preh-probbbilityy version of the one-step square root bound in Theorem Theorem 5 (High--robability one-step square-root bound). Let Q denote the conditional distribution of \ given (Z, S),, and let denote Es,s(Q]. Then, with probability at least 1 under the draw of (Z,
D(Q1I + log(
Vnn
LD L=12,8,5)-E(z.5.s) $
(18)
Again, the KL divergence can be interpreted as pointwise e-CMI, since
E Ezs,, P)] T(A;
(19)
Expressiveness of the Bounds
In Section 3, presented several new information-theorette generalization bounds, and demonstrated that they improve upon known bounds from the literature. We now turn our focus to the expressiveness of the e-CMI framework. In particular, we show that the bounds from Section 3 can be used to recover generalization guarantees meta learning from classical learning theory. Specifically, we consider the representation learning setting that is analyzed in [5]. We use the following notation. First, the sample space is the product of an instance space and label space: 2 = X' x Y. The aim of the meta learner to find a representation by E H. while the base learner outputs a ssecifcc function fw F. Composing these functions, we obtain the mapping fw hy X' + y.
4.1 Minimax Generalization Bounds
To obtain explicit minimax bounds, we assume that H has finite Natarajan dimension dx and that F has finite VC dimension dyc- This allows us to derive bounds on the entropy of the representations and predictions that the meta learner and base learner induce on the meta-supersample. This, in turn, leads to bounds the e-CMI terms that appear in the bounds in Section In the following corollary, we present the bounds that are obtained by bounding the e-CMI terms in Theorem 1 and 2. These hold for any learner that outputs hypotheses from the specified classes.
Corollary 4. Assume that the range of has cardinality N, that the Natarajan dimension of H d, and that the VC dimension of F is dyc. Also, let 2n 2 dyc + and 2n 2 dx + 1. Then,
2dv log (2)
2añ dx
2dvc log
2cm dvc
LD LD-Ī
i
(20)
7

---

Nd log
(2)
4enn dN
+ 4rddy log
2em
LD I
dyc
(21)
Corollary establishes that, for the average setting, we can the bounds in Theorems and 2 to obtain minimax bounds for function classes with bounded Natarajan and VC dimensions. Note that, the upper-bound of (20), we have fully decoupled the complexity the two function classes. This made possible by the fact that we used as the auxiliary loss in the derivation of Theorem 1, rather than L. We discuss this more detail in Appendix A.
Next, we consider the interpolating setting, where I = O. Under this assumption, we demonstrate that we can achieve better rate of convergence with respect to the number of training samples. The result, presented in the following corollary, relies on similarly bounding the e-CMI term in Corollary 3. Corollary 5. Consider the setting of Corollary 4. Furthermore, assume that L = 0. Then,
4dx log (2)
denn dx + Sńdve log
Zen
dye
LD
(22)
nn
The result in Corollary 5 demonstrates that, for the interpolating setting, the e-CMI framework expressive enough to yield bound that, ignoring logarithmic factors, decays 1/(nń), often referred to as a fast rate.
Finally, noting that the bounds in [4] and [5] are bhoh-probabilityy rather than average bounds, we also derive high-probability generalization bounds. In order achieve this, we need probabilistic upper bounds on the KL divergences that appear in Theorem 4 and 5, similar to how the e-CMI terms were bounded for Corollary 4 and 5. The resulting bounds are presented in the following corollary.
Corollary 6. Consider the setting of Corollary 4. Then, there exist constants C1, C2, C3 such that, with probability at least 1 5 under the draw of S)S.
dN log (N) & (2) dx
+log
n dvc log dyc
+oo V
8
+O21
(23)
n
dy log ((:)
nb dv
ndve log
+ log
Vnn
LD(22,,,)- S Cs
dvc
(24)
We now see that, suppressing logarithmic factors, the upper bound in (25) scales as VC(H)/á VC(F)/n, whereas the upper bound in (24) scales as /C(H)/(wñ) + C(F)/n. This matches the rates obtained by [4] and [5], respectively, demonstrating that the e-CMI framework, combined with the one-step approach, is expressive enough to recover the scaling of these results.
Note that there are some differences berween these results and the ones in [4, 5]. First, while the complexity measures that we use are related to the Natarajan and VC dimension, the results in [5] are given in terms of Gaussian complexity. Furthermore, while [4, 5] provide excess risk bounds for fixed target task with m training samples, the bounds in Corollary 6 are generalization bounds for randomly drawn task. In Section 4.2, we extend our analysis to derive excess risk bounds for fixed target task.
4.2 Excess Risk Bounds
In order to derive excess risk bounds for specific target task, as is done in [5], we need to assume that the meta learner Á and the base Iearner A empirical risk minimizers. This is in contrast to all previous bounds in this paper, which apply to any learning algorithms. Furthermore, we need notion of oracle algorithms, which minimize the population loss. Finally, we need to assume that tasks contained in the meta-supersample satisfy notion of task diversity. Intuitively, this means that, given the output of the empirical risk-min lisk-minimizing meta learner, the performance of the oracle base

---

learner on the tasks in the mute-supere gives reasonable indication of the performance of the oracle base learner on any possible task. Due to space constraints, we state here an informal version of hiigh-probability excess risk bound for a specified target task based on the one-step square-root generalization bound in Corollary 6. A precise statement of this result, along with its proof, is given in Appendix B.
2 be a vector of m samples generated independently according to the data distribution DTo for task To- Let A and A be empirical risk minimizers. Let Lo(Z, S, g denote the population loss on task To when applying Å to g and A to A(ZE) )), and let Lg denote the smallest population loss for task To that be obtained using functions from F and H. Finally, assume that the supersample satisfies a task- diversity assumption with parameters V, c. Then, there exist constants C1 and C2 such that, with probability at least 6 under the draw of (Z, S, S, D
Corollary (Informal). Consider the setting of Corollary 6 and a fixed task To- Let D
dvc log
m dvc
+ +log
y
Lo 1o(2,,,5,79)
L. S CI\
dy log
((2)
rdve log
a duc
+
log
Vnn
dx
+ C2v
te. (25)
ni
The bound in (25) displays the same scaling as the excess risk bound in [5].
It is possible derive gigh-probbbility bounds based on the two-step square-root generalization bound in Corollary 6 by suitably substituting the two-step bound in the proof of Corollary 7. The same can be done using the bounds that are given in terms of information measures, and average excess risk bounds can also be derived by an analogous procedure. Finally, we note that is possible to derive excess risk bounds for new, random task, rather than specified target task, without assuming task diversity. This is done in Corollary 8 in Appendix B.
5 Conclusions
In this paper, we derived new generalization bounds for meta learning using e-CMI, which improve upon information-thhorrticc bounds found in the literature. By considering a representation learning setting, we demonstrated that e-CMI bounds obtained via a conventional two-step approach lead to rates that coincide with those found in [4]. In contrast, we showed that by combining the e-CMI framework with one-step approach, we recover the more favourable scaling found in [5]. Note that, bounds that
while the bounds in [5] are uniform over the hypothesis class, the information-thewtt we derive are inherently algorithm- and data-dependent. As consequence, they are nonvacuous when applied to settings such classification with deep neural networks [20]. The algorithm-dependdnnc and expressiveness of our bounds indicate that they can be developed further to guide algorithm design. However, no recipe for this is provided this paper. It should also be noted that the complexity measures that we consider differ from the Gaussian complexity in [5]. An intriguing topic for further study is clarify the connection between e- e-CMI and Gaussian complexity.
Acknowledgements
This work was partly supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation and the Chalmers Al Research Center (CHAIR).
10

---

References
[1] R. Caruana. Multitask learning. Mach. Learn., 28(1):41-75, July 1997.
[2] S. Thrun and L. Pratt. Learning to Learn: Introduction and Overview. Springer, Boston, MA, USA, 1998.
[3] J. Baxter. A model of inductive bias learning. J. Artif. Int. Res., 12(1):149-198, Mar. 2000.
[4] A. Maurer, M. Pontil, and B. Romera-Paredes. The benefit of multitask representation learning.
J. Mach. Learn. Res., (1l1:28853-28844 Jan, 2016.
[5] N. Tripuraneni, M. Jordan, and C. Jin. On the theory of transfer learning: The importance of task diversity. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Vancouver, Canada, Dec. 2020.
[6] K. Lounici, M. Pontil, S. van de Geer, and A. B. Tsybakov. Oracle inequalities and optimal inference under group sparsity. The Annals of Statistics, 39(4):2164 2204, Aug. 2011.
[7] G. Cavallanti, N. Cesa- Desac-Bincchi, and C. Gentile. Linear algorithms for online multitask classifi- cation. Journal of Machine Learning Research, 11(97):2901-299 Oct. 2010.
[8] M. Pontil and A. Maurer. Excess risk bounds for multitask learning with trace norm regulariza- tion. In Proc. Conf. Learn. Theory (COLT), Princcton, NJ, USA, June 2013.
[9] L. Franceschi, P. Frasconi, S. Salzo, R. Grazzi, and M. Pontil. Bilevel programming for hyperparameter optimization and meta-learning. In Proc. Int. Conf. Mach. Learning (ICML), July 2018.
[10] M.F. Balcan, M. Khodak, and A. Talwalkar. Provable guarantees for gradient-based meta- learning. In Proc. Ini. Conf. Mach. Learning (ICML), Long Beach, CA, USA, June 2019.
[11] D. Russo and Zou. Controlling bias in adaptive data analysis using information theory. In Proc. Artif. Intell. Statist. (AISTAYS), Cadiz, Spain, May 2016.
[12] A. Xu and M. Raginsky. cnpomatiion-hhoreetee analysis of generalization capability of learning algorithms. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Long Beach, CA, USA, Dec. 2017.
[13] Y. Bu, S. Zou, and V. V. Veeravalli. Tightening mutual information-basee bounds generaliza- tion eπror. IEEE J. Sel. Areas Inf. Theory, 111::221-130 May 2020.
[14] Negrea, M. Haghifam, G. K. Dziugaite, A. Khisti, and D.M. Roy. information-theoree generalization bounds for SGLD via data-dependent estimates. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Vancouver, Canada, Dec. 2019.
[15] T. Steinke and L. Zakynthinou. Reasoning about generalization via conditional mutual informa- tion. Proc. Conf. Learn. Theory (COLT), Graz, Austria, July 2020.
[16] M. Haghifam, J. Negrea, A. Khisti, D. M. Roy, and G. K. Dziugaite. Sharpened generalization bounds based on conditional mutual information and an application to noisy, iterative algorithms. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Vancouver, Canada, Dec. 2020.
[17] B. Rodríguez-Gálvee ization error bounds and the stochastic gradient langevin dynamics algorithm. In Inf. Theory Workshop (ITW), Riva del Garda, Italy, Apr. 2020.
Bassi, R. Thobaben, and M. Skoglund. On random subset general-
[18] H. Kafez-Kolahi, Z. Golgooni, S. Kasaei, and M. Soleymani. Conditioning and processing: Techniques to improve information-theorrtie generalization bounds. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Vancouver, Canada, Dec. 2020.

---

[19] F. Hellström and G. Durisi. Data-dependent PAC-Bayesian bounds in the random-subset setting with applications to neural networks. In Workshop on Inf. Theoretc Methods Rigorous, Responsible, and Reliable Mach. Learn. (ITR3), Virtual conference, July 2021.
[20] H. Harutyunyan, M. Raginsky, Ver Steeg, and A. Galstyan. Information-theorete gener- alization bounds for black-box learning algorithms. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Virtual Conference, Dec. 2021.
[21] D. A. McAllester. Some PAC-Bayesin theorems. In Proc. Conf. Learn. Theory (COLT), Madison, WI, USA, July 1998.
[22] D.A. McAllester. A PAC-Bayesian tutorial with dropout bound. arXiv, July 2013.
[23] F. Hellström and G. Durisi. Generalization bounds via information density and conditional
information density. IEEE J. Sel. Areas Inf. Theory, 1(3):824-833 Dec. 2020.
[24] P. Alquier. User-friendly introduction to PAC-Bayes bounds. arXiv, Nov. 2021.
[25] S. T. Jose and O. Simeone. citemattiioothhooetti generalization bounds for meta-learning and applications. Entropy, 23(1), Jan. 2021.
[26] Q. Chen, C. Shui, and M. Marchand. Generalization bounds for meta-learning: An information- theoretic analysis. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Virtual Conference, Dec. 2021.
[27] S. T. Jose, O. Simeone, and G. Durisi. Transfer meta-learning: Information- theoretic bounds and information meta-risk minimization. IEEE Trans. Inf. Theor, 68(1):474-501, Jan. 2022.
[28] A. Pentina and C. Lampert. PAC-Bayesian bound for lifelong learning. Proc. Int. Conf. Mach. Learning (ICML), Bejing, China, June 2014.
[29] R. Amit and Meir. Meta-learning by adjusting priors based on extended PAC-Bayes theory. In Proc. Int. Conf. Mach. Learning (ICML), Stockholm, Sweden, July 2018.
[30] J. Rothfuss, V. Fortuin, M. Josifoski, and A. Krause. PACOH: Bayes-optimal meta-learning with PAC-guarantees. In Proc. Int. Conf. Mach. Learning (ICML), Virtual conference, July 2021.
[31] J. Guan, Z. Lu, and Y. Liu. Improved generalization risk bounds for meta-learning with PAC-Bayes-kl analysis. https: // openreview. net/ forum? id= XgS9YPYtdj. 2021.
[32] H. Flynn, D. Reeb, M. Kandemir, and J. Peters. PAC- MAC-Bayesian lifelong learning for auldi- bandits. Data Min. Knowl. Discov. 36(2):841-876, Mar. 2022.
[33] A. Farid and A. Majumdar. Generalization bounds for meta-learning via PAC-Bayes and uniform stability. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), Virtual Conference, Dec. 2021.
[34] J. Yoon, T. Kim, O. Dia, S. Kim, Y. Bengio, and S. Ahn. Bayesian model-agnostic meta-learning. In Proc. Conf. Neural Inf. Process. Syst. (NeurIPS), volume 31, Montreal, Canada, Dec. 2018.
[35] A. Rezazadeh, S. T. Jose, G. Durisi, and O. Simeone. Conditional mutual ietommtiion-bsse generalization bound for meta learning. In Proc. IEEE Int. Symp. Inf. Theory (ISIT), Melbourne, Australia, July 2021.
[36] D. Haussler and P. M. Long. A generalization of Sauer's lemma. Journal of Combinatorial Theory, Series A, 7 71(2):19-2440 1995.
[37] Y. Guermeur. Large Margin Multi-category Discriminant Models and Scale-sensitive Psi- dimensions. Research report, INRIA, Sep. 2006.
[38] M. J. Wainwright. High--iiensional Statistics: a Non-Asymptotic Viewpoint. Cambridge Univ. Press, Cambridge, U.K., 2019.
12

---

Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] The contributions section specifies where in the paper each contribution can be found.
(b) Did you describe the limitations of your work? [Yes] for our results in Section 2 and the statements of the results.
We state the assumptions required
(c) Did you discuss any potential negative societal impacts of your work? [N/A] Founda- tional work, unrelated to specific application.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] We state the assumptions required for our results in Section 2 and the statements of the results.
(b) Did you include complete proofs of all theoretical results? [Yes] Full proofs are included in the supplementary material.
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [N/A]
(b) Did you specify all the training details (e.g.. data splits, hyperparameters, how they were chosen)? [N/A]
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [N/A]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material or as URL? [N/A]
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you deseribe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent participant compensation? [N/A]
13


Reviews:
[{"review_id": "fhkCo5hya4d", "reviewer": "Reviewer_9qK1", "summary": "This paper proves new information-theoretic generalization bounds for meta-learning, using evaluated conditional mutual information (e-CMI) which was introduced by Steinke and Zakynthinou (2020) who applied it in the context of standard supervised learning. \n\nIn Section 3. The paper gives distribution- and algorithm-dependent generalization bounds based on e-CMI. Then, they show that they can use these results to recover existing bounds in the literature using CMI. Additionally, they prove a generalization bounds based on binary KL divergence. This allows them to prove “fast rate” bounds in the optimistic setting, when interpolation is possible. They also prove high-probability bounds for these three results. \n\nIn Section 4. The authors instantiate these results and show that in a representation learning type of setup, their results recover existing upper bounds based on the complexity of the class used for representations and the class used for learning task-specific predictors, but the results in this paper also seem to be more general in the sense that they don’t require any assumptions.  \n\n", "questions": "It might be good to clarify the learner and meta-learner that are used to establish Corollary 4, 5, and 6. \n\nWhat’s the term: N choose 2, that appears in Corollary 4,5, and 6? It’s not clear to me what this is just from reading the corollary statements.\n", "limitations": "It might be good to reflect on the results of the paper and discuss their implications with respect to the current practice of meta-learning. \n", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper addresses an important problem of obtaining generalization guarantees for meta-learning. I think the results provided in this paper are nice and interesting, and in particular I liked how the generic results in Section 3 can be used to recover existing results in Section 4. \n\nThe paper is well written and easy to follow. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "BQWYmP7pSwA", "reviewer": "Reviewer_7LdG", "summary": "This paper concerns the problem of understanding the generalization of the meta learning algorithms. In particular, the authors study a “two-stage” algorithm in which in the first step the meta learner learns a meta hypothesis. Then, in the second stage, based on a limited number of samples, the base learner finds the output for a particular task. \n\nThis paper extends in a very interesting way the idea of conditional mutual information (CM) from  Steinke-Zakynthinou to the meta learning setup. In particular, they consider meta-supersample instead of only supersample as in the CMI paper. Then, they prove generalization bounds based on conditional mutual information in the same spirit as in the CMI paper. Also, they provide a high probability bound in Section 4.\n\n", "questions": "- One major question is why is the generalization gap important in meta learning? Note that the results of Tripuraneni et al concern the “excess risk” of the two-stage algorithm.\n\n- The results in the paper do not give us any notion of task relatedness. It is opposed to the results in the transfer learning and meta learning literature in which we are mainly interested in answering when it is possible to do transfer learning.\n\n- Can your results tell us when meta learning is not possible? Or when is the performance of the two-stage algorithm not good?\n\n- The setup in the paper by Tripuraneni et al  is quite different from the general setup in the paper.\n\n- Corollary 4 does not represent the power of the bounds. The bounds are worst-case bounds. As stated in the intro, we are interested in the algorithmic- and distribution- dependent results.\n\n- The construction of the supersample was previously proposed in [35]. However, the bounds in this paper have better dependence on the number of samples. I think the authors should clarify the main technical differences between their proof idea and the proof idea in [35].\n\n- What is the application of the bound to more practical algorithms such as MAML?\n\n", "limitations": "The authors did not discuss the limitations of their work in the paper.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "\nI think the most interesting aspect of the paper is providing an algorithm-dependent and distribution-dependent generalization bound for the meta learning problem. Also, the bounds are not loose as shown in Section 4. \n \nRegarding weakness, the presentation of the additional structure to define the supersample structure should be improved. \nAlso, one of my concerns is that prior work on meta learning studies the “excess-risk”. However, in this work the authors study the generalization error. There is no discussion in the paper on why studying the generalization error instead of excess risk is interesting. \nThe next major weakness is that this work does not introduce any “task-relatedness” notion. \n\nAlso, the authors should spend more time on discussing the difference between their work and very related work of [35].\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "BE5BsqqDa2", "reviewer": "Reviewer_wciW", "summary": "The paper provides information-theoretic bounds for meta-learning with the recent e-CMI framework. At first, two square-root average e-CMI bounds are obtained with the two-step and one-step proof techniques, which are tighter than the previous corresponding information-theoretic bounds. Then the bounds are improved to faster rate decay ($\\frac{1}{n\\hat{n}}$) w.r.t the number of samples and number of tasks with the binary KL divergence. Next, the one-step and two-step e-CMI bounds are further extended to the high-probability setting. Finally, assuming the finite Natarajan dimension for the representation function set and finite VC dimension for the prediction function set, the expressiveness of e-CMI bounds is demonstrated to recover the scaling behaviors of meta-learning bounds obtained with classical learning theory. ", "questions": "### suggestions\nI suggest the author present Section 2 in a more logical way and simplify the notations if possible. For example, firstly introduce the original e-CMI framework that adopts supersample. Then to adapt the original e-CMI framework to meta-learning, the paper proposes the meta-supersample, which contains both ghost tasks and ghost task-specific samples, then introduce the membership, etc... \n\n### Minor issues\n\n1. Line 119 $\\mathcal{R}^{\\hat{n}}$\n2. $\\mathcal{F}$ undefined in 155\n3. Appendix equation 28 and 29, $\\gamma^2$ should be $\\frac{\\gamma^2}{2}$", "limitations": "The similarity and difference to previous Gaussian complexity bounds are mentioned when discussing the expressiveness for recovering the scaling behavior of classical learning theory. ", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n1. The paper presents theoretical results that bridge the information-theoretic meta-learning and the classical meta-learning theories. \n\n2. The author first proposes the meta-learning bounds with the e-CMI framework, which are tighter than previous mutual information bounds.\n\n3. The whole work is solid and the proof is clear. \n\n### Weaknesses\n1. The proof techniques are adapted from previous works, the one- and two-step bounds are based on previous proof ideas and the binary KL divergence bound is common in PAC Bayes theory. \n\n2. The notations are a little cumbersome. Section 2 and the description for Figure 1 are redundant and hard to follow. \n\n3. Lack of empirical validations. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept