Learning Expressive Meta-Representations with Mixture of Expert Neural Processes
Authors: Qi Wang
Herke van Hoof
Venue: NeurIPS 2022 Accept (2022)

Abstract: Neural processes (NPs) formulate exchangeable stochastic processes and are promising models for meta learning that do not require gradient updates during the testing phase. 
However, most NP variants place a strong emphasis on a global latent variable. 
This weakens the approximation power and restricts the scope of applications using NP variants, especially when data generative processes are complicated.
To resolve these issues, we propose to combine the Mixture of Expert models with Neural Processes to develop more expressive exchangeable stochastic processes, referred to as Mixture of Expert Neural Processes (MoE-NPs). 
Then we apply MoE-NPs to both few-shot supervised learning and meta reinforcement learning tasks. 
Empirical results demonstrate MoE-NPs' strong generalization capability to unseen tasks in these benchmarks.

Learning Expressive Meta-Representations with Mixture of Expert Neural Processes
Wang Amsterdam Machine Learning Lab University of Amsterdam hhg123go@gmail... com
Herke van Hoof
Amsterdam Machine Learning Lab University of Amsterdam h. vanhoof @uva. ml
Abstract
Neural processes (NPs) formulate exchangeable stochastic processes and are promising models for meta learning that do require gradient updates during the testing phase. However, most NP variants place strong emphasis on a global latent variable. This weakens the approximation power and restricts the scope of applications using NP variants, especially when data generative processes are complicated. To resolve these issues, we propose combine the Mixture of Expert models with Neural Processes to develop more expressive exchangeable stochastic processes, referred to as Mixture Expert Neural Processes (MoE-NPs). Then we apply MoE-NPs to both few-shot supervised learning and meta reinforcement learning tasks. Empirical results demonstrate MoE-NPs' strong generalization capability to unseen tasks in these benchmarks.
Introduction
Humans can naturally accommodate themselves new environ- ments after developing related skills, and this kind of adaptability relies the good abstraction of environments. Similarly, meta learning or learning to learn tries to leverage past experiences, and with help of the incorporated meta learned knowledge, a new skill can be mastered rapidly with few instances.
DC
During the past decade, an increasing number of methods have emerged in meta learning domains. In this paper, we concentrate on special branch of meta learning methods, referred to as context-based meta learning [1; 2]. representative one is a neural process (NP) [1], which was initially proposed to approx- imate Gaussian processes with lower computational cost. The core purpose of NPs is to learn meta-representations [3], which encode context points into latent variables and represent the task in functional form. In comparison to gradient-basee meta learning algorithms, e.g. woodel-ggnostie meta learning (MAML) [4], the NP directly learns functional representation and does not require additional gradient updates in fast adaptation.
x
e
21
h= 1, 1.2.....
y
Figure I: Generative Process of MoE- NPs. Here DJ refers dataset of text points in the paper. {=x] [= set of expert latent variables and is an assignment latent variable. Ob- served variables grey in circles with latent variables white.
k=1 are
Research Motivations.
Fundamentally, vanilla NPs employ
global Gaussian latent variables to specify different tasks. This setting raises several concerns in some scenarios. (i) When observations originate from mixture of stochastic processes [5], single Gaussian latent variable faced with deficiencies in formulating complex functional forms. (ii) In context-based meta reinforcement learning, the uncertainty of value functions revealed from latent variables encourages effective exploration in environments [6]. But when tasks are governed by
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

multiple variate, e.g. velocities, masses or goals in Mujoco robots [7]. the use of unimodal Gaussian latent variable restricts the flexibility of randomized value functions [8], leading sub-optimality in performance.
Developed Methods. Instead of using global latent variable in modeling, we employ multiple latent variables to induce a mixture of expert NPs to specify diverse functional priors. Meanwhile, the discrete latent variables assignment variables are introduced establish connections between a single data point and expert NPs. To optimize this model with hybrid types of latent variables, we utilize variational inference to formulate the evidence lower bound. Additionally. special modules are designed to accommodate few-shot supervised learning and meta reinforcement learning tasks.
Outline & Contributions. We overview the properties of NPs and general notations for context- based meta learning tasks in Section 3. Section 2 summarizes related work in meta learning and the NP family. In Section 4, Mixture of Expert Neural Processes (MoE-NPs)
elaborated, together with required modules for meta learning tasks. Experimental results and analysis are reported in Section Discussions and limitations are included in Conclusion Section. In principle, our contribution is two-fold:
We introduce exchangeable stochastic process, referred to as MoE-NPs, to enrich the family of NPs. Our model inherits the advantages of both mixtures of experts models and NPs, enabling multi-modal meta representations for functions.
We specify inference modules in MoE-NPs for few shot supervised learning and meta reinforcement learning tasks. Extensive meta learning experiments show that MoE-NPs can achieve competitive performance in comparison to most existing methods.
2 Literature Review
Meta Learning. Meta learning is a paradigm to enable fast learning (fast adaptation new tasks) via slow learning (meta training in the distribution of tasks). There exist several branches of meta learning algorithms. Gradient-based meta learning algorithms, e.g. MAML [4] and its variants [9; 10; 11]. perform gradient updates over model parameters to achieve fast adaptation with a few instances. cettrcs--bssd meta learning algorithms try to learn representations of tasks in distance space, and models e.g. prototypical networks [12; 13] are popular in computer vision domains. As for context-based meta learning methods of our interest, latent variable models, e.g. NPs [1], are designed to learn task representations in functional space. This family does not require gradient updates in fast adaptation.
Neural Processes Family, Apart from vanilla NPs or CNPs [2: 1]. other variants are developed and these are built on various inductive biases. To address underfitting issues, attention networks [14: 15] are introduced to augment NPs. [16; 17] improve the generalization capability of NPs with help of convolutional operations. To learn distributions of group equivariant functions, [18] has proposed EquiCNP, Similarly, SteerCNPs also incorporate equivariance to approximate stochastic fields [19]. Our work is to get NPs married with Mixture of Experts (MoEs) models [20; 21]. which model datasets with collection of expert NPs. We provide more technical summary of the NP family together with other probabilistic meta learning methods [22; 23; 24: 25] in the Appendix (F).
3 Preliminaries
Notations. The paradigm of meta learning is considered in the distribution of tasks p(T), and a task sampled from p(T) is denoted by 7 in this paper. The form of a task depends on applications. For example, a task of our interest in regressions can be function f to fit, which is a realization from unknown stochastic processes [26].
Let DC refer to a set of context points used to specify the underlying task, and DJ = [ar, vr] is set of target points to predict, e.g. f(r) yr. In the context-based meta learning with latent variables, we write the probabilistic dependencies in distributions of target functions as follows,
p(f(xr)(pf xT) = / px)p(
(1)
where the functional prior p p(21Df) is injected in modeling via latent variables

---

Neural Processes. The family of NPs [1] belongs to exchangeable stochastic processes [27]. A generative process is written as Eq. (2) with global Gaussian latent variable placed in Eq. (1),
N
PErN (VI:N) - p(z) II (i)l
(2)
where fe is mean function and of is the corresponding variance. In settings, treat the conditional neural process [2] as a special case NPs, when the distribution of a is collapsed into a Dirac delta distribution p(z) = 8(2 2) with a fixed real valued vector.
3.1 Few-Shot Supervised Learning
In the context-based meta learning, we formulate the few-shot supervised learning objective within the expected risk minimization principle follows.
min Eywp(T) [C(D; D, 0)]
(3)
The risk function L, e.g. negative fog-likelihoods, measures performance meta learning structure on the task-specific dataset D. and D. and & means parameters of common knowledge shared across tasks and parameters for fast adaptation (e.g, O denotes parameters of the encoder ø and decoder 0, and L is the approximate objective in NPs).
With the set of context points D {211 y1)..... (Im, ym)} and the target points D.T, the posterior
of global latent variable 2 in Eq. (1) is approximated with variational distribution 96 ( (D²") and an evidence lower bound (ELBO) derived to optimize in practice. general meta training objective of NPs in few-shot supervised learning is L(0, Φ) in Eq. (4), where po (D' [=) Il (v [1, =l).
(D
E, [In((P) 2 E, (=1o7)
ph Dxzluo(=1D,)) g0 (=()))))
(4)
3.2 Meta Reinforcement Learning
For the context-based meta reinforcement learning, the context points D are a set of random transition samples from environment D. = {(81, a1 82, ri),...). (8 0H 8 TH)}, where is the one-step reward after performing action at state st. Here D plays role in task inference [28] to obtain the information bottleneck (2) and D. the dataset of state action values fit.
For example, in an off-policy meta reinforcement learning algorithm, egg. PEARL [6] or FCRL [3], the general optimization objective consists of parts: (i) to approximate distributions of task-specific optimal value functions in Eq. (5), where Qo is optimal Q-value with the state value V (ii) to maximize the cumulative rewards E, [Bgg.:))pp) [R(T, 2; v)]] where R is the expected cumulative rewards in the environment 7 given policies T. (a][s,)))
[(0,0) x) = EE, (oioced rywoy (0(18,2),a) -r (r+ V((', + BE, [DKI(uo(=DP)) || p())]] (+lD)
(5)
Different from the few-shot supervised learning, here the context points are not part of fitting dataset,
which means DC o D.. . As implemented in [6], the prior distribution p(2) is typically selected as fixed one, e.g. (0, I). The induced distribution of task-specific value functions p(Qo([s, 2], a)) enables posterior sampling [29] in meta learning scenarios, which brings additional benefits of exploration for continuous control problems.
4 Model & Algorithm
In this section, we present our developed MoE-NPs and connect them to the hierarchical Bayes framework. Then approximate objectives are derived and stochastic gradient variational Bayes [30] is used to optimize the developed model. Finally, specialized neural modules are described for

---

MoE-NPs application to different meta learning tasks. We have attached concepts of variational priors and posteriors and detailed computational diagrams training and testing in Appendix (C). For the sake of simplicity, we derive equations w.r.f a task 7 in the following section, but batch of tasks are considered training implementation.
4.1 Mixture of Expert Neural Processes
Vanilla NPs often suffer underfitting in experiments [1; 14). This can be attributed expressiveness bottlenecks when employing a global latent variable in learning functional priors of tasks from unknown distributions [31]-
To alleviate mentioned deficiencies, we make two modifications for the NP family. (i) Multiple functional priors are encoded in modeling with help of K expert latent variables, which can capture statistical traits, e.g. distributional multi-modality, in data points. This setting is also an extension of Mixture of Experts (MoEs) models [20; 32; 33; 34] to meta learning scenarios. (ii) Like the gating mechanism in [5], assignment latent variables are included in modeling to select functional forms for each data point in prediction. The resulting MoE-NPs can learn more expressive functional priors and exhibit the approximation power for local properties of the dataset.
Generative Process. As displayed in Fig. (1), the graphical model involves two types of latent variables, respectively the continuous expert latent variables #1:K and the discrete assignment latent variable e. Further, we can translate the generative process into equations as follows,
K
PayiN (UI:N) = / II p(2))
II
Σ
= 1)p(ez = 1/2,, 21.k) d2
(6)
where the sampled assignment variable e is in the form of K -ddimensional one-hot encoding, and C Eq. (6) indicates the k-th expert 2 is selected from 21:K for prediction. more detailed probabilistic generative process can also be found in Appendix (E.1). In this way, our developed model constitutes an exchangeable stochastic process. And we demonstrate this claim with help of Kolmogorov Extension Theorem [35] in Appendix (E.2).
Link to Hierarchical Bayes. Note that latent variables in Eq. (6) of hybrid types. K functional priors are incorporated in expert latent variables 21:K, while the assignment latent variable is input dependent. The dependencies between 2i-K and e are reflected modeling, and this connects our work to Hierarchical Bayes [36; 37] in latent variable sense. Also when only one expert latent variable used here, the hicrarchical model degenerates to the vanilla (C)NPs [1:2] 2].
4.2 Scalable Training & Prediction
Inference Process. Given task due existence of unknown latent variables, it is intractable to perform exact inference w.t.. p (P)). As an alternative, we apply variational inference to our developed model. Here we use [z, y/ to denote single data point from set of target points DJ.
In pyllr D4) 2 Egg, [Ega.. toployla 21:K, e)] DKLllgs, (c|x, y. 21:K) || Por.2 (c|2, 21:k)]
K
(2x 9 (z DD)) C(0, のi 02)
An example for the k-th expert latent variable z5 can be in the form of a Gaussian distribution (zz pk, Et). And in meta training processes, the assignment variable is assumed to be drawn from a categorical distribution Cat(e; K, (xx y. 21:K) with parameters (a,, y. 21:x). The existence of discrete latent variables makes tough optimize using traditional methods. This is because ei- ther sampling algorithms or expectation maximization algorithms are computationally intensive when utilized here (we have discussed this point in Appendix (G)) for expert latent variables. To reduce com- putational cost, we again utilize variational inference and the decoder directly formulates the output as mixture of log- log-likell likelihoods E 1922
[In (ylp,, 21.K, e)] > k= O% ax(2,, y. #1:33 Φ2 1) In pe (yl2, =x).

---

This results in general ELBO as Eq. (7) for few-shot supervised learning in meta training, where Qo. denotes a collection of K independent variational distribution { 1 Ko..kl J. 9ds. and p respectively define the variational posterior and prior for assignment latent variables. Please refer to Appendix (C)/() for definitions and more detailed derivations.
Monte Carlo Estimates & Predictions. Meta-training processes consider a batch of tasks to optimize in iterations, and we apply Monte Carlo methods to the obtained negative ELBO C(,, di, 62) as follows.
B N
K
LMc(0,
(b) a
(b) In Inp(y,
[ .2 =
NB
B
N
DKL Los, (e,
|2 (e]"
(b)
Vi .21 || Poz. (c, (e (0)
(b) (b)
,1:k)
(8)
NB
B K ΣΣ
DKLla (2) (D)) Ionn (Df)
NB
With the number of tasks B and the number of data points in mini-batches, the Monte Carlo estimate with one stochastic forward pass is Eq. (8) for meta training objectives.
Like that in NPs [1], we derive the predictive distribution as Eq, (9) with one stochastic forward pass and parameters of discrete latent variables Poz.a (e+ I Z1:k) ) = org (a,, z1:Ki 02.2).
p(y. (V.. D.) ~ (.l1,
with 21: qor (21.K (DC)
(9)
And the point estimate prediction E|Y|X- x, D.j can also be obtained in Appendix (G.4).
4.3
Module Details for Meta Learning
4.3.1
Inference Modules in MoE-NPs
The equations so far define a framework that can be imple- mented in different meta learning tasks. Two examples are given in Fig. (2). Note that two types of latent variables are involved in modelling. need different structures of encoders for latent variables. Inference Modules are required for them, satisfying different conditions. In variational inference, distribution param- eters of these latent variables are approximated with the output of these specialized encoders.
Decodaly
Iacedara
Zacadar
Recedars,
Inference Modules for Continuous Latent Variables. For neural networks parameterize the encoder of continuous latent variables lor, we use the same architectures in (C)NPs [1; 2]. which are permutation invariant the order of context points [rc.yc] = ([T1, J.),..., [ ]- That is, for any permuta- tion operator over the set of context points, the neural network (NN) parameters of an output distribution for each expert z should satisfy jun, Σx] NNo1. ([Fo(1:N)+ ( (o).:
Maka Rrlddecceen Learain Mol--Tt
Figure 2: Computational Diagram of MoE-NPs in Meta Testing. In (a): The context variables are D [acc yc]. and the expert latent vari- ables 21.K are approximated with neu- ral networks. For discrete assignment latent variables e.. learn param- eters of categorical distributions o. with neural networks. (b): The con- text variables D are sampled tran- sitions from memory buffer. The selected expert latent variable context variable in both Actor and Critic networks during policy search.
N
Th = h1r (Iz,, vil). Th Tka juks Ex] = I (TK)
(10)
Eq. (10) is example, where h is the embedding function, + denotes a mean pooling operation, and g is part of encoder networks.
Inference Modules for Categorical Latent Variables. For neural networks to parameterize the encoder of discrete latent variables I02.1 and Poz,a need the categorical distribution parameters
5

---

to be permutation equivariant [38] with respect to the order of 21:K. This means for any order per- mutation operation o, the condition is satisfied as [an(1)), . ................................ ar(K) a2 (x, y, (o(1:)))
by = h62, (z, 2x) Vk E {1, K), [ai, 0g ......,,........................ axl = softmax(b/tt
(11)
example implementation for the variational posterior Cat(e; K axx y, 21:k) can be Eq. (11), where the vector of logits is b [b1, bg..... bxl] with t a temperature parameter. And this implementation applies to prior networks ( (o(1))) learn distribution parameters of Cat(e; K, (x, 21:K)) in the same way.
4.3.2 Meta RL Modules in MoE-NPs.
When extending MoE-NPs to meta RL tasks, optimization objectives in Eq. (7) need to be modified for Actor-Critic methods, which are employed in our settings, Like that in PEARL [6] and FCRL [3], the soft actor critic (SAC) algorithm [39] is used to learn policies due to good sample efficiency.
Given specific MDP т. posterior distributions of optimal value functions latent variables 2 in context- context-based meta RL. That is, p(Qa(s, a; M)) is approximated in the form p( (QQo[s z), a)). The resulting objectives for the Actor and Critic functions respectively in Eq. (12) and Eq. (13), where Zg is a normalization factor.
formulated via
exp (Qo([s, 2o(s)
C E BNDD? TaNTY
DEL The (a
||
(12)
The variational posterior % (2|s, DC) in Eq. (13) is state dependent distribution with V a state value function, and sampling processes refer to steps in Algorithm (3).
C = = (o.as oo [Qo([s, 2), - (r + ((s''
(13)
key difference from PEARL [6] lies that several expert latent variables and assignment latent variables involved in modeling. So refer the Kullback-Leible divergence term Eq. (14) in MoE-NPs with coefficient 30 and Bi.
FKL = 31 inna wwoo aay (rix (mf)
[D<ilaoseela 21:k) p(e)] + 80 Σ Dxu DRLlg,, (xxlD)) (x
(zx
p(=))]
(14)
The Monte Carlo estimates w.t.t Eq. (12/13/14) are used in meta training, and Pseudo code to optimize these functions is listed in Appendix (A).
Experiments and Analysis
5.1 General Settings
The implementation of MoE-NPs in meta training can found in Appendix Algorithms (1)/(), and also please refer to Appendix Algorithms (2)/(4) for the corresponding meta-testing processes. We leave the details of experimental implementations (e.g. parameters, neural architectures, correspond- ing PyTorch modules and example codes) in Appendix (H).
Baselines for Learning Tasks. Apart from MoE-NPs, methods involved in comparisons are context- based methods such as CNPs [2], NPs [1] and FCRL [3], and gradient- methods such as MAML [4] and CAVIA [9]. For FCRL, contrastive terms from SimCLR [40] are included in the objective. In meta RL, the modified NP model corresponds to PEARL [6]. Meanwhile, in Appendix (I), we include additional experimental results compared with other NPs models augmented by attentive modules [14] or convolutional modules [17].

---

5.2 Illustration in Toy Regression
To see different roles of latent variables, we visually show effects of stochastic function fitting and quantified uncertainty in toy dataset. Our goal is to discover potential components of distributions from limited observed data points.
Ground Tralh
McE-NP
CNP
NP
PCRL
The Ground Truth and Predictive Distributions of Curves using NP related Models. The gray dots around curves are the context points. The shaded regions correspond 3x standard deviations. MoE-NPs, two components the sampled mixture curve in blue and orange can be identified via assignment latent variables with more than 85% accuracy.
Figure 3:
The learning data points sampled in z--omman [-T, 3r] and merged from mixture of random- ized functions fi fz()) = sin(æ) + 6j and f2(z) = cos(xr (2) + 62 with equal probability for mixture components, where E1 (O, 0.03²) and é2 (0, 0.012). In each training iteration, sample batch of data points and randomly partition context points and target points for learning. testing phase, draw up 100 data points from this mixture of distributions with 15 random data points selected as the context. The fitting results for one sampled mixture curve are shown in Fig. (3). can be seen that both CNPs and FCRL display similar patterns, overestimate the uncertain in the mixture curve of the second component. NPs show intermediate performance and still fails to match context points well. As for MoE-NPs, with help of predicted assignment variables parameters e, one mot ( max ou.. we set the number of experts as two and partition data points to visualize predictive distributions po Ir. 2.) The MoE-NP is able to precisely separate mixture components inside the dataset and provides more reliable uncertainty.
5.3 Few-Shot Supervised Learning
We evaluate the performance of models on a system identification task in Acrobot [41] and image completion task in CIFARIO [42]. Both tasks common benchmarks in the meta learning or NPs literature [41; 43; 1; 2; 14].
Table 1: System Identification Performance in Meta Testing Acrobot Tasks. Shown are mean square errors and standard deviations in fitting meta-testing tasks. Figures in the Table are scaled by multiplying E-3 for means and standard deviations. The best results are in bold.
CNP
NP
FCRL
MAML
CAVIA
MoE-NP
2.3(+0.13) 7.2(+0.5) 2.0(+0.15) 2.5(+0.35) 2.0(+..23)
1.4(+0..6)
System Identification. For Acrobot systems, different tasks are generated by varying masses of two pendulums. A dataset of state transitions is collected by using a complete random policy to interact with sampled environments. The state consists of continuous as angles and angular velocities [0,, 82 Ø2]- The objective is predict the resulting state after selected Torque action from (-1,0, +1}. For more details about meta training dataset and environment information, refer to Appendix (H.2).
In the meta testing phase, 15 episodes with the length of horizon 200 are collected for each task and report the average predictive eπrors and standard deviations for all transitions. Here we use 50 transitions as the context points to identify the task. As exhibited in Table (I), gradient- based methods, egg. CAVIA and MAML, beat NP in terms of predictive accuracy but show higher variances than all other models. With three experts modeling. MoE-NPs significantly outperform other baselines in terms of dynamics prediction. Our finding is consistent with observations in [44], where multi-moda. distributions are necessary for Acrobot systems. We also illustrate the asymptotic performance of MoE-NPs with the increase of the context points in the following Section (5.5) Ablation part.
Image Completion. We use CIFAR 10 dataset [42] this experiment, which is formulated with 32x32 RGB images. In the meta training process, random number of pixels masked to complete
7

---

Testing in CIFAR10 Completion (Random)
Testing in CIFAR10 Completion (Ordered)
RCR.
FOK.
MANL
MAN.
0.00
OWWE
Voe
2 0.00
0.00
Figure 4: CIFARIO Complction Performance with Various Number Context Pixels. The numbers of context points used prediction are 10, 100, 200, 500, 800, 1000. The left figure is with random context pixels while the right one is with the ordered context pixels.
in images. That is, given the context pixel locations and values [xc, BC], need to learn map from each 2-D pixel location x E [o, 1 to pixel values y E IR3 Here two expert latent variables are used in MoE-NPs.
In Fig. (4), we evaluate image completion performance on the test dataset and the number of context pixels is varied in three levels. It can be found that CAVIA works best in cases with 10 random context pixels or less than 500 ordered context pixels. In other cases, MoE-NP surpasses all baselines. With more observed pixels, the predictive errors of MoE-NPs can be decreased in both random and ordered context cases. An example of image completion results is displayed in Fig. (5). For gradient-base based methods, CAVIA and MAML sensitive to the number of context points and do not exhibit asymptotic performance like that in MoE-NP. NP still suffers underfitting in performance.
5.4 Meta Reinforcement Learning
To evaluate the meta RL implementation of our model, we con- duct the experiments ments [7]. Fig. (6) exhibits the environments used in this paper, and we leave more details in Appendix (H.
2-D point robot and Mujoco environ-
Figure 5: Image Completion Visual- ization using MoE-NPs. From the Top to Bottom: the number random context pixels are 10, 500 and 1000. From the Left to the Right (every two); original images, masked images and reconstructed images.
2D Navigation. For 2-D point robot tasks, the agent needs to navigate with sparse rewards. The navigation goal of task is sampled from mixture of ares in a semi-circle in Fig. (6.a) during meta training processes.
From Fig. (7.a-c), we can observe the evaluation performance agents over iterations. For gradient-based methods, CAVIA shows better performance than MAML in exploration but both are weaker than context-based baselines. MoE-NPs can converge earlier vanilla PEARL. In particular,
with less training samples and show slight advantage asymptotic performance out of distributions (O.O.D.) tasks and show results in Fig. (7.d). We notice O.O.D. tasks are challenging for all algorithms to generalize but average returns are gradually increased with more trials. PEARL and FCRL achieve comparable rewards, while MoE-NP behaves better in this case.
test the
Locomotion. As for Half Sicetth-Coomless -irrectio (H- Cheetah-CD) and Slim Sicaanidd-CCpplee Goal (S (S-Humanoid- CG) tasks, these correspond to locomotion in complicated en- vironments. Note that multiple directions and goals are involved in tasks.
CAl Puini Rebei
do MoH-Chooas-CD
Figure 6: Environments for Meta Rein- forcement Learning. Tn (a): Blue ares are distributions of goals in orange for the robot to reach with sparse rewards. (b)/(c): Goals orange and direc- tions blue are varied in tasks.
Fig. (7) illustrates the performance of learned policies in meta learning tasks. In H-Cheetah-CD. MoE-NP shows a slight ad- vantage over FCRL, and it exhibits comparable performance in S-Humanoid-CG. In both environments, MoE-NP and FCRL outperform other basclines. This implies the importance of func-

---

(a)
(0)
(d)
Figure 7: Results in Meta Learning Continuous Control. In (a)/(M(c): Learning curves show tested average returns with variances in 4 runs. For point robol environments, 100 transitions are randomly collected from a task specific memory buffer infer the posterior. For Mujoco environments, 400 transitions collected from a task specific memory buffer to infer the posterior. In (d): Fast Adaptation Performance Meta Testing Point Robot Environments. The collected episodes are gradually increased to 50 and the average returns together with variances are visualized. goals are sampled from the white part of ares in Fig. (6.a).
randomly
tional representations for task-specific value functions. Either contrastive or multiple functional priors lead to better exploration and have a potential to boost performance in continuous control. For gradient-based methods, observations show that they can easily get stuck in the local optimal [6; 45].
5.5 Ablation Studies
Number of Experts. We examine the influence of the num- ber of experts in meta-trained MoE-NPs, and system identifi- cation in the Acrobot system is selected as an example here. As displayed in Fig. (8), the number of experts are 3, 5, 7 and 9 in different MoE-NPs. Here test the predictive per- formance of meta- meta-trained MoE-NP by varying the number of transitions. We set respectively 15, 25, 50, 100 transition sam- ples as the number of context points to identify the system. All settings for meta testing processes are already described in Section (5.3). It can be seen when the expert number is 5, the predictive performance largely enhanced with the in- crease of context points and the variance shrinks accordingly. But with more experts, e.g. greater than 5, MoE-NPs exhibit higher predictive errors, and show no significant performance improvement with the increase of the number of context points. These suggest increasing the number of experts beyond certain point can deteriorate the predictive performance of the MoE-NPs.
Meda Testory Pertemeren Aecdeer
Figure 8: Predictive Performance of MoE-NPs in Acrobot Meta Testing Processes using Varying Numbers of Expert Latent Variables and Context Points. The scale for mean square er- rors together with standard deviations is E-3.
Latent Variables in Meta RI. As mentioned in Preliminaries Section, the use of latent variable is able to induce sasccsie optimal value functions. Here we take the S-Humanoid-CC -CG as the example, and the expert encoder of MoE-NPs (Deterministic) is set to the deterministic. In Fig. (9), we observe the perfor- mance degrades lot using deterministic expert latent variables. These further verify findings in PEARL [6]. The randomness of value function distributions captures task uncertainty and encourages more efficient exploration.
2003
Mac
Sim Humanold O6
1.00
2i0
Numbbr Bicooorra
Figure 9: Ablation Porformance S- Humanoid-CC -CG. Learning curves dis- play tested average returns with vari- ances in 4 runs.
Conclusions
Technical Discussions. In this work, we have developed new variant of NP models by introducing multiple expert latent variables. Our work illustrates the roles different latent variables in MoE-NPs for meta learning tasks. MoE-NPs are able to separate data points from different clusters of stochastic processes and exhibit superior performance in few-shot supervised learning tasks. Also, MoE-NPs consistently among the best methods in meta learning continuous control.
Existing Limitations. Though the developed model provides more expressive functional priors, the appropriate number of experts is still hard to determine. Also, the mechanism of gradually

---

incorporating new expert latent variables has not been explored and this raises concerns in additional computational cost and more effective inference.
Future Extensions. Here we provide couple of heuristics to determine optimal number of experts for MoE-NPs in the future. Information metrics, e.g. Bayesian information criterion, be incorporated in modeling, Another way is to place priors over distributions of discrete latent variables like that in hierarchical Dirichlet processes [46] and select the optimal number of experts in a Bayesian way.
Acknowledgement
We thank NeurIPS anonymous reviewers and meta reviewers for their comments and constructive suggestions for our manuscript. Their engagement in the review/rebuttal period helps improve our manuscript lot.
References
[1] M. Garnelo, J. Schwarz, D. Rosenbaum, F. Viola, D. J. Rezende, S. Eslami, and Y. W. Teh, "Neural processes," arXiv preprint 187701280 2018.
[2] M. Gareeo,, D. Rosenbaum, C. Maddison, T. Ramalho, D. Saxton, M. Shanahan, Y. W. Teh,
D. Rezende, and S. A. Eslami, "Conditional neural processes," in International Conference on Machine Learning, pp. 1704-1713, PMLR, 2018.
[3] M. W. Gondal, S. Joshi, N. Rahaman, S. Bauer, M. Wuthrich, and B. Schölkopf, "Function contrastive learning of transferable meta-representationn"" in International Conference on Machine Learning, pp. 3755-3765, PMLR, 2021.
[4] C. Finn, P. Abbeel, and S. Levine, "Model-agnostic meta-learning for fast adaptation of deep networks," in International Conference on Machine Learning, pp. 1126-11333 135, PMLR, 2017.
[5] C. E. Rasmussen and Z. Ghahramani, "Infinite mixtures of gaussian process experts," Advances in neural information processing systems, vol. 2, pp. 881-888, 2002.
[6] K. Rakelly, A. Zhou, C. Finn, S. Levine, and D. Quillen, "Effieient off-policy meta- reinforcement learning via probabilistic context variables," in International conference on machine learning, pp. 5331-5340, PMLR, 2019.
[7] E. Todorov. T. Erez, and Y. Tassa, "Mujoco: physics engine for model-based control," in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 0026-5033, IEEE, 2012.
[8] Osband, B. Van Roy, and Z. Wen, "Generalization and exploration via randomized value functions," in International Conference on Machine Learning, pp. 2377-2386, PMLR, 2016.
[9] L. Zintgraf, K. Shiarli, V. Kurin, K. Hofmann, and S. Whiteson, "Fast context adaptation via meta-learning." " in International Conference on Machine Learning, pp. 7693-7702, PMLR, 2019.
[10] A. Rajeswaran, C. Finn, S. Kakade, and S. Levine, "Meta-leaming with implicit gradients," 2019.
[11] C. Finn, K. Xu, and Levine, "Probabilistic agnel-ag meta-learning," in NewrIPS, 2018.
[12] Snell, K. Swersky, and Zemel, "Prototypical networks for few-shot learning." in Pro- ceedings of the 31st International Conference on Neural Information Processing Systems, pp. 4080-4090, 2017.
[13] K. Allen, Shelhamer, Shin, and J. Tenenbaum, "Infinite mixture prototypes for few-shot learning." in International Conference on Machine Learning, pp. 232 241, PMLR, 2019.
[14] H. Kim, A. Mnih, J. Schwarz, M. Garnelo, A. Eslami, D. Rosenbaum, O. Vinyals, and Y. W. Teh, "Attentive neural processes," in International Conference on Learning Representations, 2019.
10

---

[15] M. Kim, K. R. Go, and S.-Y. Yun, "Neural processes with stochastic attention: Paying more attention to the context dataset," in International Conference on Learning Representations. 2021.
[16] A. Y. Foong, W. Bruinsma. J. Gordon, Y. Dubois, J. Requeima, and R. E. Turner, "Meta-learning stationary stochastic process prediction with convolutional neural processes," " in NewrIPS, 2020.
[17] J. Gordon, W. P. Bruinsma, A. Y. Foong. Requeima, Y. Dubois, and R. E. Turner, "Convolu- tional conditional neural processes," in International Conference Learning Representations, 2020.
[18] M. Kawano, W. Kumagai, A. Sannai, Y. Iwasawa, and Y. Matsuo, "Group equivariant conditional neural processes," in International Conference Learning Representations, 2020.
[19] P. Holderrieth, M. J. Hutchinson, and Y. W. Teh, "Equivariant learning of stochastic fields: Gaussian processes and steerable conditional neural processes," in International Conference on Machine Learning, pp. 4297-4307 PMLR, 2021.
[20] L. Xu, M. Jordan, and G. E. Hinton, "An alternative model for mixtures of experts," Advances in neural information processing systems, pp. 633-640, 1995.
[21] S. Waterhouse, D. MacKay, T. Robinson, et al., "Bayesian methods for mixtures of experts," Advances in neural information processing systems, pp. 351-357, 1996.
[22] Gordon, J. Bronskill, M. Bauer, S. Nowozin, and R. Turner, "Meta-lcarning probabilistic inference for prediction," in International Conference on Learning Representations, 2018.
[23] E. Iakovleva, J. Verbeek, and K. Alahari, "Meta-learning with shared amortized variational inference," in International Conference on Machine Learning, pp. 4572-45822 PMLR, 2020.
[24] Requeima, Gordon, Bronskill, Nowozin, and E. Turner, "Fast and flexible multi-task classification using conditional neural adaptive processes," Advances in Neural Information Processing Systems, vol. 32, 2019.
[25] Z. Sun, Wu, X. Li, W. Yang. and J.-H. Xue, "Amortized bayesian prototype meta-learning: new probabilistic meta-learning approach to few-shot image c " in International Conference on Artificial Intelligence and Statistics, pp. 1414-1422, PMLR, 2021.
[26] C. E. Rasmussen, "Gaussian processes in machine learning," 1 in Summer school on machine learning, pp. 63-71, Springer, 2003.
[27] M. Ross, J. J. Kelly, R. J. Sullivan, J. Perry, D. Mercer, R. M. Davis, T. D. Washburn, E. V. Sager, J. B. Boyce, and V. L. Bristow, Stochastic processes, vol. Wiley New York, 1996.
[28] J. Humplik, A. Galashov, L. Hasenclever, P. A. Ortega, Y. W. Teh, and N. Heess, "Meta reinforcement learning as task inference." arXiv preprini arXiv: 06424, 2019.
[29] D. Russo and B. Van Roy, "Learning to optimize via posterior sampling." Mathematics of Operations Research, vol. 39, no. 4, pp. 1221-1243, 2014.
[30] D. P. Kingma and M. Welling, "Auto-encoding variational bayes," arXiv preprint 13126:1312.6114, 2013.
[31] N. Dilokthanakul, A. Mediano, M. Garnelo, M. C. Lee, H. Salimbeni, K. Arulkumaran, and M. Shanahan, "Deep unsupervised clustering with gaussian mixture variational autoencoders." 2016.
arXiv preprint
[32] N. Shazeer, A. Mirhoseini, Maziarz, Davis, Q. Le, G. Hinton, and J. Dean, "Outrageously large neural networks: The sparsely- gated mixture-of-expers layer," 2016.
[33] S. E. Yuksel, J. N. Wilson, and P. D. Gader, "Twenty years of mixture of experts," IEEE transactions on neural networks and learning systems, vol. 23, no. 8, pp. 1177-1193, 2012.
[34] Pei, P. Ren, C. Monz, and de Rijke, "Retrospective and prospective mixture-of--eenratoos for task-oriented dialogue response generation," in ECAI 2020, pp. 2148-2155, IOS Press, 2020.

---

[35] R. N. Bhattacharya and E. C. Waymire, Stochastic processes with applications. SIAM, 2009.
[36] N. D. Lawrence and J. C. Platt, "Learning to learn with the informative vector machine," in Proceedings of the twenty-first international conference on Machine learning, p. 65, 2004.
[37] H. Daumé III. "Bayesian multitask learning with latent hierarchies," " in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 135-142, 2009.
[38] M. Finzi, M. Welling, and A. G. Wilson, "A practical method for constructing equivariant multilayer perceptrons for arbitrary matrix groups," arXiv preprint 1rXiv:2104.09455: 2021.
[39] Haarnoja, A. Zhou, P. Abbeel, and S. Levine, "Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with stochastic actor," in International conference on machine learning, pp. 1861-18700 PMLR, 2018.
[40] Chen, S. Kornblith, M. Norouzi, and G. Hinton, "A simple framework for contrastive learning of visual representations," in International conference on machine learning, pp. 1597-1607, PMLR, 2020.
[41] T. W. Killian, S. Daulton, G. Konidaris, and F. Doshi Velez, "Robust and efficient transfer learning with hidden parameter markov decision processes," Advances in neural information processing systems, vol. 30, 2017.
[42] A. Krizhevsky, G. Hinton, et al., "Learning multiple layers of features from tiny images." " 2009.
[43] A. Galashov, J. Schwarz, H. Kim, M. Garnelo, D. Saxton, Kohli, S. Eslami, and Y. W. Teh, "Meta-learning surrogate models for sequential decision making," arXiv preprint ar:ivv /997, 1907, 2019.
[44] B. Kégl, G. Hurtado, and A. Thomas, "Model-based micro-data reinforcement learning: what are the crucial model properties and which model to choose?," arXiv preprint arXiv:2107.11558 2021.
[45] Li, Q. Vuong, S. Liu, M. Liu, K. Ciosek, K. Ross, H. Christensen, and H. Su, "Multi-task batch reinforcement learning with metric learning," arXiv preprint arXiv: 1909. 1373, 2019.
[46] Y. W. Teh, M. I. Jordan, M. J. Beal, and D. M. Blei, "Hierarchical dirichlet processes," Journal of the american statistical association, vol. 101, no. 476, pp. 1566-1581, 2006.
[47] M. Volpp, F. Flürenbrock, L. Grossberger, C. Daniel, and G. Neumann, "Bayesian context aggregation for neural processes," in International Conference on Learning Representations, 2020.
[48] A. Eslami, D. Jimenez Rezende, Besse, F. Viola, A. S. Morcos, Garnelo, Ruderman, A. A. Rusu, I. Danihelka, K. Gregor, et al., "Neural scene representation and rendering." Science, vol. 360, no. 6394, pp. 1210-121, 2018
[49] D. Lepikhin, H. Lee, Y. Xu, Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen, "Gshard: Scaling giant models with conditional computation and automatic sharding," arXiv preprint arXiv:2006. 16668, 2020.
[50] S. Rajbhannari,, C. Li, Z. Yao, M. Zhang, R. Y. Aminabadi, Awan, J. Rasley, and Y. He, "Deepspeed-mos Advancing mixture-of-expert inference and training to power next-generation scale," " arXiv preprint arXiv:2201. 220552200 2022.
[51] Y. Shi, Paige, Torr, et al., "Variational mixture-of-experts autoencoders for multi-modal deep generative models," Advances in Neural Information Processing Systems, vol. 32, 2019,
[52] P. Kidger, J. Morrill, J. Foster, and T. Lyons, "Neural controlled differential equations for irregular time series," Advances Neural Information Processing Systems, vol. 33, pp. 6696- 6707, 2020.
[53] M. Schirmer, M. Eltayeb, S. Lessmann, and M. Rudolph, "Modeling irregular time series with continuous recurrent units," International Conference on Machine Learning, pp. 19388- 19405, PMLR, 2022.
12

---

[54] E. Jang, S. Gu, and B. Poole, "Categorical reparameterization with preprint arXiv:1611.01 144, 2016
arXiv
[55] K. Sohn, H. Lee, and X. Yan, "Learning structured output representation using deep conditional generative models," Advances in neural information processing systems, vol. 28, 2015.
[56] B. De Finetti, "La prévision: ses lois logiques, sources subjectives," in Annales de I l'institut Henri Poincaré, vol. 7, pp. 1-68, 1937.
[57] H. Edwards and A. Storkey, "Towards a neural statistician," 2017.
[58] Q. Wang and H. Van Hoof, "Doubly stochastic variational inference for neural processes with hierarchical latent variables," in International Conference on Machine Learning, pp. 10018- 10028, PMLR, 2020.
[59] C. Louizos, X. Shi, K. Schutte, and M. Welling, "The functional neural process," Advances in Neural Information Prucessing Systems, vol. 32, pp. 8746-8757, 2019.
[60] E. Mathieu, A. Foster, and W. Teh, "On contrastive representations of stochastic processes," arXiv preprint arXiv:2106. 10052, 2021.
[61] Yoon, G. Singh, and Ahn, "Robustifying sequential neural processes," in International Conference on Machine Learning, pp. 10861-10870, PMLR, 2020.
[62] T. K. Moon, "The expectation-maxiiizaiin algorithm,"" IEEE Signal processing magazine, vol. 13, no. 6, pp. 47-60, 1996.
[63] E. J. Gumbel, Statistical theory of extreme values and some practical applications: a series of lectures, vol. 33. US Government Printing Office, 1954.
[64] R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement rentectonnieenninerceeennnt learning," Machine learning, vol. 8, no. 3, pp. 229-256, 1992.
[65] H. Ren, A. Garg, and A. Anandkumar, "Contextbased meta-reinforreemet learning with struc- tured latent space," in Skills Workshop NeurIPS, 2019.
[66] A. Le, H. Kim, M. Garnelo, D. Rosenbaum, Schwarz, and W. Teh, "Empirical evaluation of neural process objectives," in NeurIPS workshop on Bayesian Deep Learning, p. 71, 2018.
[67] J. Schulman, S. Levine, P. Abbeel, M. Jordan, and P. Moritz, "Trust region policy optimization," in International conference on machine learning, pp. 1889-1897, PMLR, 2015.
Checklist
The checklist follows the references. Please read the checklist guidelines carefully for information on how to answer these questions. For each question, change the default [TODO] [Yes]. [No], or [N/A] You are strongly encouraged to include justification to your answer, either by referencing the appropriate section of your paper or providing a brief inline description. For example:
Did you include the license to the code and datasets? [Yes] See Section ??.
Did you include the license to the code and datasets? [No] The code and the data are proprictary.
Did you include the license to the code and datasets? [N/A]
Please do not modify the questions and only use the provided macros for your answers. Note that the
Checklist section does not count towards the page limit. In your paper, please delete this instructions block and only keep the Checklist section heading above along with the questions/anssers below.
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
13

---

(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (c.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) your work uses existing assets, did you eite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're re using/curating? [Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "kAkBSDNHILA", "reviewer": "Reviewer_et6m", "summary": "The paper has a very good motivation of combining neural processes (NP) and a mixture of experts (MoE) with the hope to further strengthen the power of NP as MoE usually does with other architectures in the application of context-based meta-learning. To enable a collection of experts, a single global latent variable is replaced by a collection of latent variables, which represent MoE NPs. ", "questions": "Line 66/67 seems to be overly stated, implying gradient-based optimization is slower than NP, which might not be true. Gaussian Processes have cubic cost without proper modification, and NP marries GP and NN to inherit the fast training. Consequently, it is not clear whether NP is strictly faster than NN, at least based on this hybrid intuition. Please elaborate or explain why your claim is so. \n\n\nIt seems for simpler tasks, the model performs really well but with more difficult tasks, the variance is much higher (Fig 7), is it because of variational and Monte Carlo approximations over so many latent variables? \n\n\nSimilar to the scalability point in Weakness above: although the authors already presented a high performance in difficult settings, e.g. Humannoid, it would be more practical for this solution if you could also demonstrate a higher-scale experiment in which there are a big number of tasks, maybe meta-learning for multilingual MT. \n\n\nSee also in Weakness above. \n", "limitations": "N/A", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength \n1. Apart from motivation, the paper has clear explanations of step-by-step derivations.\n2. Experiments show strong improvements over the chosen baselines. \n\nWeakness\n1. One important question in practice is scalability. Naturally expanded from NP, it would be good to compare it with at least NP since the model is more and more complex as more latent variables are injected. Also, it’s not clear (as also mentioned in 5.5) why an increasing number of experts would not help after 5? \n\n- So, I would suggest doing some expert-specific studies about mean/variances, learned representation, and system-related statistics to see the difference between them. One hypothesis is that the model collapses somewhere that only a few experts actually learn and thus increasing the number of experts would not help. This can vary by tasks/data distributions as well. \n\n- Similarly for Latent Variables ablation in 5.5, I would suggest doing a per-task, per-expert analysis to see if it matches the intuition on whether each expert is specialized for each (group of) task(s), which would further solidify the intuition to replace a global variable with different local variables.   \n\n2. Despite a very good motivation and setting, there might be a need to compare with some other gradient-based MoE performance because MoE naturally enhances models’ complexity (as in the same case here with the introduction of more latent variables) for more power, in exchange for being more complex and other costs such as communication and/or memory. Likewise, it would be good if we also have some comparison with some other MoE baselines such as GShard, Deepspeed MoE, …\n\n3. No code is provided. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}, {"review_id": "VgEDV5Uf19b", "reviewer": "Reviewer_fZgG", "summary": "This paper proposes a new Neural Processes family, Mixture of Expert Neural Processes (MoE-NPs). While NPs represent each task as a single vector representation sampled from Gaussian distribution, MoE-NPs use discrete representation to select which representation will be used for each data. Therefore, MoE-NPs have more flexibility than other NPs and it is shown in this paper empirically from 1D sinusoidal regression task to Meta Reinforcement Learning tasks.\n\n\n==========================\nThe authors addressed my concerns enough, I increase my score.", "questions": "- In Table 4, ANPs results are same for the different number of given contexts. Is it right? or typo?\n- As described in the above session, I am not sure when MoE-NPs is strong. When we need to use different $z$ in a task?\n- In section 5.2, the amplitudes or phases of $sin$ and $cos$ function, is it right?\n", "limitations": "I think the authors already addressed the limitations of their model adequately, but more clarifying why it is required for some cases will make this paper more concrete.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- MoE-NPs take the idea of Mixture of Expert to NPs and show better expressiveness than other NPs.\n- Equations are derived in appendix, which helps to understand.\n- Including ANP and Conv-NP, they evaluated MoE-NPs with a wide range of baselines comprehensively.\n\nWeaknesses\n- I can understand MoE-NPs can have more representative power than NPs, but I am not sure when we need it. The toy regression task is to show the strength of this method, but except that, I cannot find the killer applications for MoE-NPs. MoE-NPs shows more stable results than others for image completion in Figure 4, but the y-axis range is quite small, so I am not sure we can say MoE-NPs is necessary for that. For RL, MoE-NP didn't show a significant performance gap from baselines.\n- In Figure 8, it is also weakness that overestimated experts hurt the performance. I think it is because by representing a single underlying function with more than 1 expert, the model misses the underlying patterns.\n- In section 4.2, notations are quite confusing (e.g., $q_{\\pi 2,1}$ and $p_{\\pi 2,2}$ are the variational posterior and prior for categorical variable, why $(2,1)$ and $(2,2)$?). They are not well described before this section, so it is hard to understand. \n- For toy regression task (section 5.2), two functions look no variations without the noise. For example, in [1], they made variations on amplitudes or phase, but as described in section 5.2, this task doesn't have those variations to learn the meta-learning framework. I don't think that it is enough to validate those meta-learning models.\n\n[1] Finn, Chelsea, Pieter Abbeel, and Sergey Levine. \"Model-agnostic meta-learning for fast adaptation of deep networks.\" International conference on machine learning. PMLR, 2017.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "HENRZLOXCuY", "reviewer": "Reviewer_ZLhe", "summary": "This paper studies the problem of learning-to-learn a new task $\\tau$ from a dataset $D_{\\tau}$ of labeled pairs $(x_1, y_1), ..., (x_n, y_n)$. The authors propose to do so using meta learning with neural processes (NPs), trained via variational inference. They propose a new type of NP model that uses a multi-modal mixture density as a prior, called Mixture of Expert Neural Processes (MoE-NPs).\n\nThe problem with vanilla NPs is that they use a single gaussian latent variable z to specify an entire task. As result they have limited expressiveness when trying to learn functional priors over tasks from unknown distributions. The authors propose to resolve this issue by using a mixture prior with K gaussian distributions and latent variables $z_{1:K}, e$ where $e$ is a categorical mixture assignment variable.", "questions": "There is an important choice that has to be made for the MoE-NP model, which is the number of mixtures K. In all of the experiments the authors take this number as a given (K=3 for Acrobot, K=2 for CIFAR-10), but I assume there is a lot of tuning and optimization that goes on behind the scenes to settle upon a value that works well. I would suggest that the authors say a little bit about this tuning procedure and what it looked like for the experiments.\n\nPretty much all of the implementation details of the model are left to the appendix, which is non-standard. I suggest finding some room in the main paper to say at least a little bit about the implementation you're working with.", "limitations": "Limitations are adequately addressed", "rating": 7, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper writing is generally clear and thorough. Although I was unfamiliar with NPs before reading the paper, I believe that the proposed extension provides an original and significant contribution to the field based on the provided theory and experiments.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept