Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This work introduces a  novel PAC-Bayes bound for gradient-based meta-learning algorithms. This PAC-Bayes bound exploits a previous bound for uniformly stable learning algorithms.  In opposite to previously proposed bounds for meta-learning methods, the bound is non-vacuous in some data sets and much tighter. Authors also show that a heuristic method based on the minimization of the proposed bound leads to novel meta-learning algorithms. 
main_review: The theoretical study of meta-learning algorithms is a very relevant topic in machine learning due to the wide range of potential applications. This work proposes a novel PAC-Bayes bounds that builds in two different frameworks to compute performance guarantees: PAC-Bayes theory and uniformly stable algorithms. Although the uniformly stability bound was previously proposed, this connection seems novel to me. 

The paper is very well written and easy to follow despite the technical difficulty of the topic. And the proposed bound is thoroughly evaluated and gives rise to a novel class of bounds that are non-vacuous in some relevant cases. The derivation and evaluation of a novel meta-learning algorithm based on the minimization of the proposed PAC-Bayes bound are also worth noticing. 
limitations_and_societal_impact: The limitations of the paper are discussed as well as the societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a generalization bound for meta-learning based on the following PAC frameworks: Uniform Stability for assessing the generalization of the base (task-specific) models to unseen data, and PAC-Bayes for estimating the generalization of the meta-learner to unseen tasks.
This implies the use of a stochastic meta-learner, that provides the distribution from which base models are sampled for initializing the task learner.
The obtained bound is then tightened for the few-shot learning context, by accounting for additional validation data.
The derived bound is finally used as objective function for optimizing the posterior model distribution by stochastic gradient descent, for the particular case of Gaussian posterior and prior distributions and using the reparameterization trick. 
main_review: The paper's contributions build on works on bi-level generalization guarantees. To the best of my knowledge, the combination of uniform stability and PAC-Bayes frameworks is new in the meta-learning literature.

The paper is generally well written, but two key points should be clarified:

1. The notion of uniform stability is used to derive a generalization guarantee in expectation, contrary to the original result from Bousquet which is a PAC guarantee, hence standing with a certain probability. 
This is a key aspect of the approach and should be highlighted, as it allows to derive the bound of Theorem 3 for meta-learning without using a union bound. Indeed, as highlighted in the paper relying on a union bound for deriving the generalization guarantees is a disadvantage of existing works, as it results in looser guarantees.
I would also suggest to explicitly report in the theorems the parameters on which \beta_{US} depends, in order to make it easier for the reader to assess the bound. 

2. In Algorithm 1, the posterior's mean and variance are updated via a Monte Carlo estimate of the expected loss, by drawing a model per task to then update it with the base-learner algorithm and to be used to estimate the bound. However I believe that this procedure is unbiased only if the base-learner algorithm is differentiable and jointly optimized with the posterior. It is not clear if this is the case in the proposed optimization procedure.

The main downside of the derived bounds is that the convergence of the PAC-Bayesian complexity terms explicitly depends only on the number of tasks and not on the sizes of the task samples. This is a source of vacuity on real problems, as the number of tasks is usually small. 
Indeed, in the experiments with the Omniglot dataset of the paper, the complexity term has to be reduced by using smaller models and by down-scaling it.
A simple (but limited) improvement could be achieved by using PAC-Bayesian bounds with faster convergence rate, such as Seeger's bound [Seeger][Maurer].

Finally, in the experimental evaluation, it would be useful to report also the test error of a model trained on each task singularly (averaged over all the tasks). This would be a valuable sanity check, to make sure that the proposed meta-learning algorithm does not degrade the baseline performance.

### Minor:
In Table 2 PAC-BUS bound increases when the number of points m increases. Shouldn't it be the reverse?

[Seeger] PAC-Bayesian Generalisation Error Bounds for Gaussian Process Classification, JMLR, 2002.

[Maurer] A note on the PAC Bayesian theorem, 2004

### UPDATE
The authors addressed well my questions, but after reading also the other reviews I am still not sure whether recommending acceptance or rejection of the paper, and I am keeping my initial rating of 6. The work improves upon state-of-the-art generalization guarantees for meta-learning and in particular in the few-shot learning context and the paper is well written. However, I still have the following concerns:

1. the novelty of the contributions is limited, as the proposed bound is a modification of works by [Pentina et al.] using (known) stability results at the base-level instead of a PAC-Bayes guarantee;
2. the practical impact of the bound itself is also limited as at the meta-level it depends only on the number of tasks; this is an intrinsic problem of bi-level guarantees, and the solutions proposed by the authors can only marginally alleviate it.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper provides generalization bounds for meta-learning approaches. The result combines two frameworks: uniform stability of gradient descent based learning algorithms that cover guarantees at the base classifiers level and PAC-Bayes theory for obtaining results at the meta level. The objective is to have bounds that become tighter when the base learner adapts quickly. The framework proposed allows the authors to develop new algorithms and regularization scheme. An experimental evaluation is conducted on a toy problem and standard benchmarks.
main_review: Originality
---------
Understanding fewshot learning methods is still challenging and results going in this direction are important. The proposed contribution combines an existing uniform stability framework with a classic version of the PAC-Bayes framework. This is not particular new, the novelty lies here in the application to the fewshot learning framework. 
In my opinion, the novelty is moderate.



Quality
------
The global scope is of ok. The authors have completed their claims by complementary results in Appendix.
I have however the following reservations:
-The uniform stability framework of (Hardt et al., ICML'16) is initially presented for randomized algorithms, so the validity of the contributions for non randomized models has to be justified. See the remarks in the comments below.
-The PAC-Bayes term of Eq.(7) includes only the number tasks and does not include the size of the samples.
-Comparisons to over PB-based frameworks would have strengthened the experimental part.


Clarity
------
The paper is globally clear.
Few additional lines for describing a bit clearly the fewshot framework (meta-learning, meta-testing, k-way N-shot) would certainly help an unfamiliar reader to better understand some concepts presented in the paper.



Significance
----------------
The basis of the work relies on existing techniques. The result obtained in Thm 3 is relatively direct from the considered frameworks. The contribution represents another step in considering PAC-Bayes for few shot learning, but the result in my opinion is not particularly significant.
In particular, the result does not exhibits how the combination between tasks and the number of instances can lead to learn with few examples: even with the increasing number of training data, the number of tasks still need to be important. From the bound, it seems that the number of tasks considered should be very important.    

From the practical evaluation, the bounds seem informative, but comparisons to other settings would also be helpful. I guess here that the quality of $\Theta_0$ is essential and I did not see much details on how to find this initialization, which would also be interesting for reproducing the results.

Thm1 and deduced results has to be refined with respect to the correct framework.


Comments
------------

-About Thm1: If I am not mistaken, the result in Thm1 coming from (Hardt et al., ICML'16 [22]) is valid for randomized algorithms and then an expectation over A must be added in the definition. Same for Equation 4. So the application to non stochastic algorithms must be better justified.    

In the paper, the authors consider theequivalent  following definition of stability    
$$
sup_{S,S^i} sup_z sup_\theta [L(h_A(\theta,S), z) - L(h_A(\theta,S^i), z)] \le \beta_{US},    
$$

while, the definition of (Hardt et al., ICML'16 [22]) corresponds to   
$$
sup_{S,S^i} sup_z E_\theta [L(h_A(\theta,S), z) - L(h_A(\theta,S^i), z)] \le \beta_{US} 
$$

The definition considered in the paper implies the definition of (Hardt et al., ICML'16 [22]). I hope this is what the authors expect, but at least a comment on the change of definition should be made.   

However, in Thm 2.2 of (Hardt et al., ICML'16 [22]), it is shown that   
$$
 E_{S,\theta} L(h_{A(\theta,S)}, T) \le E_{S,\theta} L(h_{A(\theta,S)}, S) + \beta_{US}.   
$$

So actually, an expectation is missing.   

This expectation is missing when one wants also to prove Thm3. That being said, a more formal justification of Thm3 (i.e. with a proof) could be useful maybe.   

The validity of the approach for gradient descent (and not SGD) should also be discussed accordingly.   


-Thm2 uses the classic McAllester version. It could be interesting to obtain other frameworks to obtain tighter results that may be more informative for the few-shot learning context

-Equation 7: here, I guess we assume that each sample S_i has the same number of examples. As far as I can understand, the result, the number of examples is not involved in the PAC-Bayes term, which makes this result similar to frameworks using an hyperprior model. 


-In Table 1: the only PAC-Bayes bound comes from reference [3], but it would be interesting to compare to other PAC-Bayes results.

-l.223-226, distribution D_{S_{ev}} is introduced here but the justification of this distribution is a bit fuzzy. Few additional lines to explain it would be welcomed here.


-About the result. Other recent meta-learning frameworks have tried to propose bounds in O(l\times (m+n)) [Du et al., Few-Shot Learning via Learning the Representation, Provably, 2020; Tripuraneni et al., Provable Meta-Learning of Linear Representations, 2020].  This could be a strong result to obtain such bounds in a Pac-Bayes context. 

I also would have appreciated a discussion on the expressiveness of the framework with respect to other PB-settings for fewshot/meta-learning learning, beyond the computational issues: what is more general, less/more expressive, can the proposed framework be derived from existing frameworks by choosing particular hyperprior for example?

-Lines 272-273 "Notably, this problem is challenging for losses which are convex in the parameters of the network (since the decision boundary between classes is nonlinear)" - I am not sure to understand the formulation in general, since what matters is non linearity and not especially non-convexity. It maybe easier to have non-linear formulations with non-convex models, but the formulation should be rewritten here.

-In the related work, the authors should consider and discuss the following reference that is one the first mixing the concepts of stability and PAC-Bayes:   
*Ben London: A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descent. NeurIPS 2017.    
This is not exactly the same setting, but this paper is important regarding the topic

-In appendix, lines 18-23, about the fact to use the training data is necessary to have guarantees on performance. I may have missed something, but this points seems to require more justification, in particular why not having another validation set is not sufficient? The learned models become independent from S. The bound might be over-optimistic.
Then for the validity of the PB bound this raises questions because the optimization of \theta should be justified to not depend on S.


-----------  
After rebuttal  
-----------  
The authors did a very important work in order to bring more justification and additional information.
From the feedbacks provided, I am now convinced that the work is correct and of quality.
Adding a study on the impact of the parameters of algorithm on the bound and the behavior of the model is for me very interesting in the context of the work.
The paper needs now to be updated according to the comments provided to improve the clarity of the paper.
The reservations on the work, I have are that the potential increasing of the bound when training data increases and the possibility to use large validation sets in a few-shot learning contexts, anyway in a multi-task setting this point would be less important.

limitations_and_societal_impact: The authors discuss some limitations of the approaches in the conclusion, in particular the convergence properties of the algorithms provided.
They also provide a section on the broader impact of the work.

The work proposed in this paper is mainly fundamental and does not imply some direct negative societal impact. Actually, by trying to provide some generalization guarantees, the papers contributes to a better general understanding of the field.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This submission is about generalization bounds for gradient-based meta-learning algorithms. It is motivated by the observations that meta-learning algorithms have demonstrated promising empirical results in a variety of tasks, while our theoretical understanding of these algorithms is lagging behind and in particular the existing generalization bounds for meta-learning are challenging to compute or give vacuous values even in simple settings.

I like how this work reasons about the challenges in generalization for meta-learning. The reasoning is by separately considering generalization at the base level and generalization at the meta level, and how the two levels must be coupled. The work proposes to rely on uniform stability to reason about generalization at the base level, and to use PAC-Bayes bounds to reason about generalization at the meta level. The claimed main contribution is that this approach leads to a novel generalization bound (Theorem 3) for gradient-based meta-learning, which is then leveraged to develop a learning method (PAC-BUS) by using the bound as optimization objective, and the method is evaluated in two meta-learning problems. It is not too clear to me how the two levels are coupled to learn the inductive biases that can be used for future tasks, but I suspect that the PAC-Bayes prior must play a role in connecting the two levels.

The argument behind the proof of the main result (Theorem 3) appears to consist of three blocks: (1) writing the distribution that generates data sets of a given size $m$ as a product distribution between the random tasks and the random samples for given tasks; (2) using the classical PAC-Bayes bound of McAllester for bounding the expected loss (expectation over data sets and over parameters) in terms of its empirical counterpart (that sums over the training tasks) plus a KL regularization term; and (3) using Uniform Stability for bounding the expected loss on a given task in terms of its empirical counterpart and stability. However, I missed an outline of the proof or indication of where the reader can see the full details of the proof.
main_review: ORIGINALITY

Generalization bounds for meta-learning have been proposed before. There are stability-based approaches (works of Maurer) and approaches that are based on PAC-Bayes bounds (Amit and Meir). However, the combination of these two approaches seems to be a novel idea.

SIGNIFICANCE

Assuming that the main result is correct, this is an interesting result conceptually and potentially also with respect to the improved tightness of the numerical bound values that it may lead to in practice. However, I would like to flag that the details of the proof are not provided, nor is the proof outlined at least. I would like to request the author(s) to include in their response an outline of the proof of Theorem 3, and in case this paper is accepted, the author(s) should include such outlined proof in the paper and an additional appendix giving the full details of the proof.

CLARITY / QUALITY / READABILITY

The paper is relatively well-written. The introduction is very nice and informative. However, I have some concerns regarding Section 2 on problem formulation, where the mathematical language is declared. It seems to me that the choices of notation affect the clarity/readability of the paper in a non-trivial way. I have made an effort to point out specific difficulties that I encountered and to suggest possible fixes. Many other comments of editorial nature are shared next.

EDITORIAL FEEDBACK

Title: Would the authors consider the title "Meta-Learning Bounds via PAC-Bayes and Uniform Stability" ?
(The acronym PAC-BUS can be introduced and used in the paper.)

L.8-9: "the result of this approach is a novel PAC bound [...]" (i.e. insert "of this approach" and remove the hyphen)

L.19: Replace "one" with "a learner" 

L.23: "base learner" (no hyphen, I think)

L.36: Replace "realistic problems" with "benchmark problems" and also delete "ImageNet-scale"

L.43: "meta-learner" (twice)

L.48: Together with [11] it would be good to cite Bousquet et al. (2020) "Sharper bounds for uniformly stable algorithms"

L.50-51: I think the definition of "uniform stability" of Bousquet & Eliseeff [11] is in the sup norm (hence it makes sense that they called it "uniform"), while the definition of Hardt et al. [22] is in expectation (the L_1 norm of the difference) only. Since these two definitions do not coincide, this passage is potentially confusing/misleading without further clarification. Perhaps say that [22] demonstrated that limiting the number of training epochs [...] leads to stability in expectation.

L.52: delete "uniformly"
L.54: delete "uniform"

Possibly the work of Shalev-Shwartz and collaborators ("Learnability, stability and uniform convergence" and "Learnability and stability in the general learning setting") and perhaps work of Poggio and collaborators ("On learnability, complexity and stability") could be cited to support the connection between stability and learnability, on which you rely at the base level?

L.60-61: Replace "one has seen" with "that have been seen"

L.68-69: "We refer to the resulting approach as PAC-BUS since it combines PAC-Bayes and Uniform Stability to derive generalization guarantees for meta-learning."

Section 2: The math notation choice has an impact on the clarity/readability of the submission. I suggest to choose notation that helps keep concepts clearly distinguishable but at the same time is suggestive in that it makes apparent the connection between related objects, and overall notation that is convenient and easy to follow. As reference I suggest looking e.g. the papers of Andreas Maurer ("Algorithmic Stability and Meta-Learning" and "A Second-order Look at Stability and Generalization") or the statistical learning literature (papers/surveys by Massart, Boucheron, Lugosi and others) which I consider good examples with regards to the mathematical language that notation that hey use. In particular, I recommend the convention of using capitals (e.g. $X$ or $Y$ or $Z$) to denote random variables, using their lower-case counterparts ($x$, $y$, $z$ respectively) to denote arbitrary possible values of these random variables, and using calligraphic capitals ($\mathcal{X}$, $\mathcal{Y}$, $\mathcal{Z}$) for the sets of all possible values of these variables (i.e. their "spaces"). It makes perfectly clear the distinction between the different categories of objects (a random quantity, a fixed but arbitrary quantity, or the set of all possible values, respectively). I also recommend a separate kind of notation for the distributions of random variables, like $P$ (for "probability") or $\mu$ (must come from "measure" and is used often to denote a probability measure), and using subscripts for different $P$'s or $\mu$'s (e.g. $P_t$ or $\mu_t$ for the distribution of task $t$). Then if needed you could choose suitable notation for the set of all probability distributions over a given space, and so on. 

Last note on notation choice: It could be good to make the distinction between expected loss and empirical loss visually apparent, say $L(h,\mu_t)$ and $\hat{L}(h,S_t)$ where the "hat" in the latter indicates the sample average.

L.124-125: "More recent frameworks include PAC-Bayes theory [...]" (since the acronym has been declared before)

L.127: " [49, 31, 19, 44]" (i.e. reference [44] belongs in this context as well, as do the works of Parrado-Hernandez et al. (2012) "PAC-Bayes bounds with data dependent priors" and Ambrolarze et al. (2006) "Tighter PAC-Bayes bounds" arguably.)

L.128: " [31, 33, 41]" (i.e. reference [41] is relevant here, which supersedes [45] with respect to tightening the bound values) and for the works that generalize the framework perhaps the list [13, 14] should be enlarged with Rivasplata et al. (2020) "PAC-Bayes Analysis Beyond the usual Bounds" at least.

Note that there is an important difference between a citation like [27] for "FMTL" and a citation like [23] for "FTL" since in the former case "FMTL" was introduced in the given citation [27] while in the latter case "FTL" existed in the literature before [23]. Hence, to make this difference apparent, I suggest writing "see e.g. [23]" if you want to use [23] as a reference for FTL. I am not objecting [23] as a suitable reference to look up the definition of FTL, in fact I believe it is, what I am suggesting is to use "see e.g." in order to signal that [23] is a good place to look up "FTL" while there are (many) others. Please make sure this distinction is followed for other references throughout the paper.

L.176: Replace "by some" with "by a copy" and clarify the distributional assumption of the $z'_i$ for the sake of clarity. Unless these data points are not random. But if $z_1, ..., z_m$ are random, as it sees from the problem formulation (L.86), then it makes sense that   $z'_1, ..., z'_m$ should also be random, and probably they are independent from $z_1, ..., z_m$ but coming from the same distribution.

L.180-182 (Definition 1): I don't see the point of "differ-by-one" (nor the corresponding acronym) since there is no other notion of change in the dataset in your paper (i.e. there is no "differ-by-two" or "differ-by-three"), On the other hand, as is well known, analysing stability to changing two or three data points reduces to analysing change in one data point at a time by the triangle inequality, hence the stability to changing one datapoint is the fundamental notion here. I prefer to read "Algorithm $A$ is uniformly stable with $\beta>0$ with respect to a loss $L$ if ..." and then after Eq.(2) write that you define $\beta_{US}$ to be the minimal $\beta>0$ in the right hand side. Since it is specific to algorithm $A$, probably it is even better to define $\beta_{US}(A)$ as the minimal such $\beta$. There is something missing in the quantification of variables in this definition, since $z'_i$ are currently not quantified. Also, if this condition is for random $z_1, ..., z_m$ and $z'_1, ..., z'_m$ (i.e. they are random variables) then probably the requirement should be that the inequality holds almost surely. But it looks that your definition wants to require that the inequality holds for any arbitrary $z_1, ..., z_m$ and $z'_1, ..., z'_m$, in which case the notation choice is hot helping to distinguish random variables from the arbitrary values that they may take. [A notation choice that I recommend is using capital letters (e.g. $X$) for a random variable, the corresponding lower-case ($x$) for an arbitrary possible value of this random variable, and capital calligraphic ($\mathcal{X}$) for the space of all possible values of this random variable. If used consistently, this notation makes it crystal clear what category of object occurs at each place.] In case (as I suspect) this definition is meant to require the inequality for all arbitrary choices of $z_1, ..., z_m$ and $z'_1, ..., z'_m$, it could be more convenient to write Eq.(2) with a supremum on the left-hand side, and clarifying in the next line that the supremum is over all $m$-tuples $(z_1, ..., z_m)$ and $(z'_1, ..., z'_m)$ that differ at one entry, and over the other things ($z$ and $i$ and $\theta$). Also I'd like to point out that the role of the task $t$ is not apparent in Eq. (2) currently, which needs clarification.

L.183-184: Since there is no citation to which this Theorem 1 is attributed, it could be assumed that you are implying that this theorem is yours? Could be better to write "The following result [22, Theorem X.Y] establishes a relationship between uniform stability and generalization in expectation." (Note that I am suggesting to be explicit about which theorem in [22] you are citing.)

L.188: This is a good place to discus the outline of the proof of Theorem 1, and insert a cross-reference to where the proof is given in full detail. 

L.196: Is "sample $s$" a single datapoint? Ir is $s$ a list of datapoints? This needs to be clarified. Correspondingly, the distribution $\mathcal{D}$ should be over the space of a single data point or over some product space for tuples of data points, this needs to be said explicitly for the sake of clarity.

L.199: "the following holds simultaneously for all" (i.e. insert "simultaneously")

Note that the right hand side of Eq.(6) shows that you are using the form of McAllester's bound with the sharp dependence on the number of examples $l$, and this sharp dependence is due to Maurer (2004) "A note on the PAC-Bayesian theorem" (McAllester's bound was similar but with a mildly weaker dependence on the number of examples). On this note, the bound of McAllester that you are implying by Eq.(6) was due to McAllester (1999) "PAC-Bayesian Model Averaging" (COLT 1999), while the reference [34] in your list of references is McAllester (1998) "Some PAC-Bayesian Theorems" (it appeared first in COLT 1998, and then next year it also appeared in the journal Machine Learning). The point is that the correct year for [34] is 1998. But the bound that you are implying in Eq.(6) is the one given in "PAC-Bayesian Model Averaging" (with the sharp dependence on $l$ as per Maurer (2004)).

L.206: I miss the point of defining this marginal distribution $\mathcal{D}_S$ over datasets of size $m$. Should not such marginal distribution be $\mathcal{D}^m$ (i.e. the product of $m$ copies of the distribution $\mathcal{D}$)?

L.209: This line is puzzling. In PAC-Bayes bounds the quantity being bounded is an expectation of a loss of the form $L(h_{\theta},S)$ where the expectation is over the distribution of $\theta$ and over the distribution of $S$. In your Eq.(7) the two arguments of this loss are coupled, as both depend on $S$. Please elaborate on how to reason about this? There are some conditional expectations that have not been made explicit? Does the PAC-Bayes bound still hold in this case?

L.215: The assumption is that $0 \leq L(h,z) \leq 1$ for any $h$ in the hypothesis space for the given problem, and any $z$ in the example space. The current formulation is unnecessarily overloaded.

L.238-239: The notation "$s$" was used for a sample in the previous page, while here in this page it is used for the marginal variance parameters of a Gaussian with diagonal covariance. Could we use a different symbol? Also, it would be good to be explicit about the dimensionality of $\theta$ and then it will be clear the dimension of the space over which we have these Gaussian distributions.

L.241: Explain the symbol $\odot$ used in Eq.(10) which I assume is a Hadamard product?

L.251-252: Could the authors provide an appendix with details for the extension to mini-batches of tasks?

L.252-253: "stochastic gradient descent (SGD) and gradient descent (GD)"

Caption of Table 1: "We also report the meta-test loss (CEL_s) for all methods."
Also, may we see the definition of the acronym CEL (and a suitable reference)?

L.275: "cross-entropy" (with hyphen)

L.284: Was this notation declared? May be I missed it, but it would not harm to clarify (write explicit) that N is the neural network and  the norm with subscript F is the Frobenious norm.

L.335-336: Insert a reference to support the claim that "it may cause memorization" which refers to learning the prior, was this something reported by [58]? Or if this was something that was observed in your own experiments, then explain this fact with accompanying discussion.

L.345: "meta-learning: PAC-BUS" (i.e. replace the comma with a colon)

L.367: What are "broader PAC-Bayes bounds"? I did not follow the meaning of "broader" in this sentence (while the meaning of "tighter" is clear, of course).

L.368-369: "using a PAC-Bayes bound for unbounded loss functions for the meta-generalization step"


L.296: "through the continuous-bag-of-words GloVe embedding [38] into dimension d = 50 to generate samples"

Would like for all my feedback above to be addressed in the author response.
limitations_and_societal_impact: Sure.
review_title: null

Meta-review: The reviewers and myself all agree on the significance of the results and their interest to the NeurIPS community. The reviews expressed a number of technical concerns which were all addressed satisfactorily by the authors during the discussion phase. If not done already, I would invite the authors to consider revising their manuscript along the many suggestions made by the reviewers.
Decision: Accept (Poster)