Neural Bootstrapping Attention for Neural Processes
Authors: Minsub Lee
Junhyun Park
Sojin Jang
Chanhui Lee
Hyungjoo Cho
Minsuk Shin
Sungbin Lim
Venue: ICLR 2022 Submitted (2022)
Keywords: Neural Process
Bootstrapping

Abstract: Neural Processes (NP) learn to fit a broad class of stochastic processes with neural networks. Modeling functional uncertainty is an important aspect of learning stochastic processes. Recently, Bootstrapping (Attentive) Neural Processes (B(A)NP) propose a bootstrap method to capture the functional uncertainty which can replace the latent variable in (Attentive) Neural Processes ((A)NP), thus overcoming the limitations of Gaussian assumption on the latent variable. However, B(A)NP conduct bootstrapping in a non-parallelizable and memory-inefficient way and fail to capture diverse patterns in the stochastic processes. Furthermore, we found that ANP and BANP both tend to overfit in some cases. To resolve these problems, we propose an efficient and easy-to-implement approach, Neural Bootstrapping Attentive Neural Processes (NeuBANP). NeuBANP learns to generate the bootstrap distribution of random functions by injecting multiple random weights into the encoder and the loss function. We evaluate our models in benchmark experiments including Bayesian optimization and contextual multi-armed bandit. NeuBANP achieves state-of-the-art performance in both of the sequential decision-making tasks, and this empirically shows that our method greatly improves the quality of functional uncertainty modeling.

Under review as a conference paper at ICLR 2022
NEURAL BOOTSTRAPPING ATTENTION FOR NEURAL PROCESSES
Anonymous authors Paper under double-blind review
ABSTRACT
Neural Processes learn to fit a broad class of stochastic processes with neural ner- works. Modeling functional uncertainty is important aspect of learning stochastic processes. Recently, Bootstrapping Neural Processes (B(A)NP) propose bootstrap method capture the functional uncertainty which can replace the latent variable (Attentive) Neural Processes ((A)NP), thus overcoming the limitations of Gaussian assumption on the latent variable. However, B(A)NP conduct bootstrapping in a non-parallelizable and memory-inefficien way and fail to capture diverse patterns in the stochastic processes. Furthermore, we found that ANP and BANP both tend to overfit in some cases. To resolve these problems, we propose an efficient and easy-to-imp teyyllt-mmpleeent approach, Neural Bootstrapping Attentive Neural Processes (NeuBANP). NeuBANP learns to generate the bootstrap distribution of random func- tions by injecting multiple random weights into the encoder and the loss function. We evaluate our models in benchmark experiments including Bayesian optimization and contextual sequential
bandit. NeuBANP achieves the best performance in the tasks among NP methods, and this empirically shows that our method greatly improves the quality of functional uncertainty modeling.
INTRODUCTION
Neural Processes (NP) (Garnelo et al., 2018b) define distributions over functions given a set of observations, and are trained via a meta-learning framework so that it can adapt to new functions rapidly. NP learns to model a wide range stochastic processes and can estimate the uncertainty over the predictions with less computational effort, compared to Gaussian Processes (GP) (Rasmussen, 2003). However, NP frequently suffers from a fundamental drawback of underfitting. As a remedy of the underfitting issue, Attentive Neural Processes (ANP) (Kim et al., 201 8) applies the attention modules to the encoder network. Despite this modification, a single Gaussian latent variable of (A)NP inducingfuncctinal has limitation in inducing, functional uncertainty (Louizos al., 2019), global uncertainty that
decides the distribution over the space of trajectories or functions.
Appropriate modeling of functional uncertainty in stochastic processes improves the predictive performance and diversity in function realizations (Le et al., 2018). thus provides principled way to guide agents to find optimal candidates in sequential decision-makeer problems. In these tasks, model needs to approximate a function and estimate uncertainty correctly to optimize a black-box box function whose analytic information not given. Although GP widely used for these tasks, these are the promising area for the application of NP because GP is computationally expensive, and it can be hard to choose an appropriate prior. Recently, Bootstrapping Neural Processes (B(A)NP) (Lee et al., 2020) modify (A)NP to induce more robust uncertainty estimation by employing the residual bootstrapping. However, B NP underperforms in capturing a functional uncertainty because the residual bootstrapping works in homoscedastic way, removing the connection between the feature and the label in its bootstrapped samples. The bootstrap strategy used in B B(A)NP demands a higher computational burden compared (A)NP since it requires multiple computations of the encoder network and additional heuristics, including the adaptation layer and the lower bound on the variance Furthermore, ANP and BANP tend overfit for a simple regression task rather than learning the underlying heteroscedasticity. We observed that this problem is a by-product of the attention modules
We colored the revised or added sentences in blue only for the rebuttal. This color will not appear in the final version of the manuscript.

---

Under review as a conference paper at ICLR 2022
used in both models (see Figure 1 and 5). This finding suggests that effective regularization of attention modules is required to prevent overfitting. Additionally,, as explained in Section 4, ANP and BANP often tend to estimate homogeneous uncertainty regardless of whether the observation is given.
ANP
BANP
NeuBANP
-15
Fig. 1. Each plot shows predictions given by ANP, BANP, and the proposed method in linear regression. The ground-trute truth function simple linear function with heterogeneous variance: y + Be(z) where <(x) ~ ((, o² (x)) and (z) = vx + 10-3, See Appendix B.I for more details.
To resolve the problems of previous NPS, we introduce novel bootstrapping method for neural processes, Neural Bootstrapping Attentive Neural Processes (NeuBANP). Motivated from the recent work on efficient bootstrapping of the neural network, Neural Bootstrapper (NeuBoots) (Shin et al., 2021), we introduce bootstrapping of the attention in a computationally efficient way by simple modification of the input of attention modules and the loss function, instead of memory-inefficient resampling and contrived heuristics employed in BANP. The simplicity and computational efficiency of NeuBANP directly come from NeuBoots, but does not guarantee the performance in modeling the functional uncertainty since NeuBANP operates on the meta-learning framework. Thus, we modify the method for training the model to learn the randomness present in the underlying function, allowing the model to estimate random functions generated by any stochastic process. This modification is simple but gives a strong consistency between the bootstrapped samples and the representations from the attention modules, unlike the residual bootstrapping used in BANP. Besides, our bootstrapping method, which implements the concatenation and multiplication of random bootstrap weights, operates as regularizer on the attention networks, thus preventing overfitted predictions observed in previous attention-baa based NP methods. NeuBANP is trained to generate a valid predictive distribution by utilizing the uncertainty inherent in observations by bootstrapping instead of the uncertainty that depends on the prior assumption on the latent variable as (A)NP. This leads to the success in capturing heteroscedasticity of the data and modeling functional uncertainty in stochastic processes. As result, NeuBANP achieves the best performance in sequential decision- -akinn problems such as Bayesian optimization and contextual multi-armed bandit. The experimental results demonstrate that our model provides promising capabilities as an efficient neural approximation of stochastic processes.
Contributions We propose NeuBANP, an easy-to-implement and computationally efficient method for bootstrapping ANP. The proposed method has a novelty in learning a generator for bootstrapping stochastic processes under meta-learning framework. NeuBANP resolves overfitting problem of attention modules and shows robust performance on heteroscedastic models without heuristics like the extra adaptation layer in BANP. NeuBANP estimates functional uncertainty better than BANP and achieves the best performance in stochastic optimization problems, including multi-dimensional Bayesian optimization and contextual multi-armed bandit, compared to previous methods.
2 PRELIMINARIES
2.1 META- FRAMEWORK NEURAL PROCESSES
Consider data D = (X (z,, ni) W) C X' x,, the pairs of inputs x, E X' and outputs y/ E y.
Let P be a probability distribution over functions of E F: y, = f(x,) + e; where G>~N(0,) of), hence P determines the distribution of D. For disjoint index sets C and T satisfying CUT = [n].
define context De = (Xc, Yc) =
B.) {( hoec and target Df = (X7, Y7) = {(x,, heee, so
that D=
De U Dr. The task to learn the neural processes po that fits the stochastic processes
2

---

Under review as a conference paper at ICLR 2022
P given De when C Pn, is randomly chosen subset of [n], follows:
0_ argmin Ei-p P. log Po (Y (Y
0
This meta-learning framework allows pe to learn diverse patterns in JF via prior distribution P; hence NP can predict target points conditioned on contexts adaptively the inference phase.
2.2 (BOOTSTRAPPING) ATTENTIVE NEURAL PROCESSES
ANP ANP is variant of NP equipped with attention modules in the encoder part. See Appendix for the detailed definition of attention operations. Let the context Dee is given, and the model aims to infer for given feature
X. The encoder network of ANP maps (I,, De) into pair of representation vectors r = (z, h) as follows:
{selcEc = SelfAttn(De)) (effAttt(De),
z (=22))((ss)) CrossAttt(z,, Xe, (heleec)
(2)
{heleEe
(3)
Here, u4= and Ø= are single linear layers that map Se mean and standard deviation of the latent variable z, respectively. In ANP, z and h refer to latent path and deterministic path, respectively. The deterministic path models the overall skeleton of the encoder network, while the latent path models the functional uncertainty using a stochastic global latent variable (Garnelo et al., 2018 2018b; Kim et al., 2018). Then the decoder network takes the representations and target data æ as imputs predict u(x,r) and or (z, r), the parameters of the conditional predictive distribution p(y x. De) = N (vlp, o²).
BANP The global latent variable in ANP potentially limits the flexibility in expressing functional uncertainty. BANP proposes a method that utilizes paired bootstrapping and residual bootstrapping model stochasticity in a data-driven way. First, they resample pairs of (æe, ye) with replacement to construct D
e v-(h), Here b= = 1...., B implies the number of bootstrap samples. The resampled context is encoded into the representation vector f(b) (2 (b)) by replacing De in (2) and (3) by (()) Then BANP conducts the residual bootstrapping to construct bootstrapped contexts again D (b) = (Xc y-(")). ). By using De and (())) in (2) and (3) separately, BANP gets the representations contexts (r, f(b)). Finally, BANP uses an adaptation layer to merge (r, f(6)) and obtain (b) (b) through the decoder network (see Figure 2).
ply,, r, (()) = N (y) (0)))) Plylx, Dc)
B = plyl 1,yl,,r,if (., F(0)).
E b=1
(4)
Due to the residual bootstrapping, BANP conducts the encoder computation three times and the decoder computation twice for a single forward propagation. These additional calculations cause the computational bottleneck in the training and inference (see Appendix B.6).
2.3 NEURAL BOOTSTRAPPER
Repetitions of training restrain the practical use of bootstrap procedures in deep neural networks due to their high computational burden. To alleviate this, Shin et al. (2021) proposes Neural Bootstrapper (NeuBoots). NeuBoots circumvents multiple training of networks by learning bootstrap generator.
Random Weight Bootstrapping Let Eeec {(f(x)) Ue) be the loss function of interest for neural network f. standard bootstrap procedures, bootstrapped neural network can be obtained by minimizing the loss function weighted by random bootstrap weight we == {we : c E CJ:
L(we, f. Dc) = Σ wwlllf w.el((xx))
(5)
GEC
According to the choice the distribution on We, various bootstrap procedures can be represented under the form of (5): e.g., the paired bootstrap by we Multinomial(n; 1 1/n...., 1/n) and the Random Weight Bootstrapping (RWB) (Prestgaard & Wellner, 1993; Newton & Raftery, 1994) by we |C| x Mtrichlet(L...... 1). NeuBoots utilizes RWB avoid the data discard problem which can occur in the standard bootstrapping. We then compute bootstrapped neural networks (B)
( b I,..., B) via minimization of (5) for sampled w(!)) We
3

---

Under review as a conference paper at ICLR 2022
Learning To Generate Bootstrap Distribution The main idea of NeuBoots is construct single generative network that models the bootstrapped neural networks with varying bootstrap weights in (5). This formulation modifies the backbone network in a form of f(z, we) that inputs both feature and bootstrap weight we. Shin et al. (2021) show that the minimizer of the following loss generates valid bootstrap evaluations that match the results of the standard bootstrap procedure:
[(F De) = X Dirichles( [L(we: f(,, we), Dc))],
We call this weighted bootstrapping loss. Once this generator is trained via single optimization procedure, we can efficiently generate bootstrapped predictions by plugging random bootstrap weights in the trained generator; i.e., for feature of interest æ, the trained generator inputs { e /8=1 and produces bootstrapped predictions g(b) f(x, w (!)) for = I....,.. B.
(b) B
NEURAL BOOTSTRAPPING ATTENTIVE NEURAL PROCESSES
We propose a novel class of NP, called Neural Bootstrapping Attentive Neural Processes (NeuBANP).
Aligned with the previous formulation of NP families, we can define our model po as:
/ P h
/II 22, h, Dc)dz
Po (Y(,)
(6)
II (uilari, h, 2) where z ~ ((z(Dc).
(7)
Here Ps is the decoder and q denotes the posterior distribution. Since the above integral is intractable,
we approximate the predictive distribution by sampling z from the bootstrap distribution q(-1Dc). in-
stead of Gaussian distribution as in (A)NP. Precisely, we train a generative encoder network Jø. which
outputs bootstrapped representation pairs (h, z) = {(h(), 2 Through the meta-learning framework (1), our model learns to generate bootstrapped predictions for arbitrarily given function f ~~7 P. Thus, our approach can be regarded as a learn-to-bootsiee method for stochastic processes. Compared to the fixed bootstrap method used in BANP, our Iearnable bootstrap method can find the best strategy to appropriately generate the random functions regarding the given context and the general property of the underlying data generating process. This will lead to the better modeling of functional uncertainty of the target stochastic processes. Also, note that NeuBANP can obtain a number of bootstrapped predictions by simply plugging different bootstrap weights into 9a. while BANP needs repetitive data resampling from scratch. Figure 2 shows the difference between the forward computation paths of BANP and NeuBANP in detail.
(X, De, {w( (b) 1 1b=1) )
3.1 NEURAL BOOTSTRAPPING CONTEXTS WITH ATTENTION MODULES
To train generative network, which outputs the bootstrapped representations, we modify the encoder in ANP to take both (x, Dc) and bootstrap weight we inputs. We introduce the posterior and the prediction paths in the encoder, analogous to NP's latent and deterministic paths. NeuBANP is designed to leverage RWB, which is theoretically proven posterior and prediction path in NeuBANP take the random bootstrap weight as an auxiliary input. This input provides enough randomness into the network that model can successfully capture the functional uncertainty.
valid bootstrap method. In detail, the
Posterior path We tag each context (Io, Me) E De with bootstrap weight wh!) to construct bootstrapped contexts D( = {xe. ye, wa Jcecc. Then the posterior path receives ((!) as input and outputs latent variable (b)), In detail, apply self-attention to D(0) and multiply the resultant representation (b) by the bootstrap weight w before mean aggregation. This path connects the con- texts with weighted bootstrapping loss during the training and allows the bootstrap weights model bootstrapped posterior distribution by controlling the magnitude of each context representation.
Prediction path The prediction path outputs a target-specific representation h (b) that is relevant for the prediction. We apply self-attention to contexts De and multiply the bootstrap weight w(O)
4

---

Under review as a conference paper at ICLR 2022
Encoder
X
Croua Actention
C
valle
c
His
Self Acsent
hc
Xe
Decoder
Araptation
Decoder
Suie
2
rondaa
boutauep
(a) BANP
Encoder
ML.P
Ceuoa Attention
h
Elenetwise produc
Paodicion pads
YP
5o11
hg
X
Decoder
cppp
Beanrogying
[C] x Dirichlet(1. 1)
7C
Actertion
(b) NeuBANP
Fig. 2. A single forward computation of (a) BANP and (b) NeuBANP. Note that the inference for each target point requires B times of this forward computation.
element- element-wisely to compute another bootstrapped representation. Cross-attention module uses Xc and this bootstrapped representation as key-value pairs to which the target query x attends. Consequently, reflecting the bootstrap weights shared with the posterior path, the prediction path models interactions between the given context and the target input.
We summarize the above posterior and prediction paths as follows:
(b) = T = (D) (,),,, z (b) = mean WC (0)
c
(8)
be (heloec SelfAun(Pe). hb) CrossAttm Xc, be @ (
(9)
where O denotes element-wise multiplication. We concatenate random bootstrap weights to context data, unlike NeuBoots, which only utilizes random weight multiplication in the final layer. Concatena- tion of weights and the contexts in the posterior path yields two effects. First, concatenating random information in given contexts provides sufficient randomness to the model, allowing it to cover a wide range of function samples in a space where true function likely to exist. Second, we made this modification to maximize the use of random weights and provide bootstrapping information to the model without resampling the data, enabling better uncertainty estimation. Additionally, by multiplying the random weights to the representations from both paths, as in (8) and (9), the repre- sentations are consistent with the weights and maximize bootstrapping effect. We think that these multiplications also provide the effect of regularization, which mitigates the overfitting tendency which ANP and BANP show in a simple regression experiment (see Figure/ 1).
Decoder and Bootsrapped Prediction The decoder takes (z (b) (b)) from the encoder and target a X as inputs to generate a prediction أ(و)- MLP(x, (b) h (bi). We can generate bootstrap sam-
ples ((1), f(() ..., O) by plugging We we ...., w(
(B) into the encoder (x, De, -), respectively. We estimate the predictive mean and standard deviation using bootstrap samples:
(1) (2)
B B
B Σ
(b(b) 6
11)
(10)
B
In the previous NP methods, the decoder directly outputs the parameters of the predictive distribution, but the decoder of NeuBANP outputs the stochastic predictions (f(6)) 1o-1 This design of output is a distinction of our model from other NP methods and allows the nonparametric estimation. Due to this structure, existing NPS the lower bound of standard deviation for robust performance.
5

---

Under review as a conference paper at ICLR 2022
We found that the performance susceptible to the lower bound value. However, NeuBANP naturally obtains parameters of predictive distribution using bootstrap predictions and shows better performance without such heuristics. It also has the advantage of calculating higher-order statistics without changing the structure.
3.2 TRAINING
Weighted Bootstrapping Loss We train NeuBANP with weighted loss similar that of NeuBoots as demonstrated in Section 2.3. Only context data has the corresponding bootstrap weight in our setting, but the model still has fit target data. Thus we designed loss function as a sum of weighted context loss Leomext and non-weighted target loss Liargtt follows:
1 B
Liotal
(h log N (YelMes oB))
log (og.((yel (1I)
T
CEC
IET
Lorge
where Me; des M4, and dg computed by (10) for context (Iee Ve) De and target (It, Me) Dry Averaging weighted context loss with multiple bootstrap weights improves the robustness model by showing various bootstrap samples during training. We trained the model with negative log-likelihood (NLI).. One can replace the NLI with a different loss function, such as cross-entropy according to the target tasks.
EXPERIMENTS
We conducted experiments to compare NeuBANP with the previous methods for regression and sequential decision-making problems. For regression tasks, we conducted onc-dimensional (ID) regression experiments on random functions generated from GP prior and image completion tasks as two-dimensional (2D) regression (see Appendix B.5 for image completion). For sequential decision-makee making problems, we evaluated each method in Bayesian optimization (BO) and Contextual Multi-Armed Bandit (CMAB).
4.1 NONPARAMETRIC REGRESSION AND UNCERTAINTY ESTIMATION
Settings We followed the settings in Lee et al. (2020). To obtain meta- veta-training datasets, we sampled batches of random functions from prior with RBF kernel, and context and target points were chosen randomly from each function. In addition, kernel parameters of GP were randomly sampled so that the models could learn about various functions. NeuBANP was trained with 10 bootstrap samples, and we confirmed that it is robust to the number of samples. Please refer to Appendix B.2 for details.
Results The numerical results are summarized Table 2. ANP and BANP tend to estimate homo- geneous uncertainties for all target points in situation where context points are sufficiently given (see Figure 3), because they place the heuristic lower bound on the variance. In the case of BANP, homoscedasticity occurs BANP, which is that does not properly estimate functional uncertainty. Since the uncertainty about the shape of the true function is high in the region where the context point is not given, models should generate a wide range of function samples. However, we can see that BANP generates almost the same functions, like those of the other methods, for modeling functional uncertainty. NeuB ANP resolves these problems efficiently and achieves the best performance except for target prediction in the Periodic kernel.
when the number of context is small. Another problem only occurs in
argue that BANP is not appropriate method
4.2 BAYESIAN OPTIMIZATION
Since NP can approximate class of arbitrary functions, it can replace GP, the commonly used surrogate model of Bo. is crucial to approximate the objective function from the given observations using the surrogate model, but evaluating the acquisition function and determining the subsequent samples are vital for efficient exploration. We evaluated the proposed method on various block-box functions, which may be unobserved in the meta-training step (see Algorithm 1).

---

Under review as a conference paper at ICLR 2022
ANP
BANP
NeuBANP
0.00
0.0
0.03
-2
-2
-1
0.50
0.50
0.50
0.2-
0.25
0.25
0.00
0.00
0.50
-0.0
-350
Fig. 3. Comparison of ANP, BANP, and NeuBANP in ID regression given 4 context points (top) and 20 context points (bottom). Orange lines represent the ground-truth function. Blue lines are predictive mean given by each model and shaded region denotes standard deviation (amount uncertainty). To visualize quality of functional uncertainty, we overlapped multiple shaded areas obtained with 30 sampled outputs for each input.
Settings For ID, we followed the same setting in Lee et (2020). We set objective functions gen- erated from GP with RBF, Matérn 5/2, and Periodic kernels and applied the models trained in Section 4.1. Furthermore, we demonstrated the BO performance of NeuBANP for (2D and 3D). We set objective functions various benchmark functions used in the optimization literature (Kim, 2020; Kim Choi, 2017). See Appendix B.3 for details. simple regret measured the performance of each model, and the mean performance over 100 experiments is reported for reliable evaluations.
settings
20 Ackley
2D Dropwave
Michelewicz
Time complexity
05
T5mm
02
3D Ackley
Cosine
3D Rastrigin
Time complexity
11
135
170
030
47S
evalcatid paints
Fig. 4. Left: Multi-dimensional Bayesian optimization results on various benchmark functions with UCB as an acquisition function. Bold lines represent the mean performance over 100 experiments. We indicate 20% of the standard deviation. Right most: Time complexity of each model as the number of observations increases.
Results NeuBANP outperformed the other methods BO experiments (see Figure 4 and 8). Table and 4 shows numerical results. For multi-dimmesionall BO, NeuBANP achieved the best results for every function except the Hartmann-3D when using Upper Confidence Bound (UCB) as an acquisition function. For the Hartmann-3DD, the performance NeuBANP is statistically comparable the best method (ANP) when considering the standard deviation. For Goldsteinprice and Rastrigin functions, GP records poor performance due to its numerical errors during the optimization procedure (see Appendix B.3). The rightmost plots of Figure 4 show the time complexity according to the number of observation points. NeuBANP has the fastest decreasing rate regrets in terms of iterations in both cases. We also conducted performance NeuBANP to be independent of the selection acquisition functions (Figure 9). Figure 10 demonstrates the better exploration strategy of NeuBANP compared BANP. The second
experiments using Expected Improvement (EI) to test the

---

Under review as a conference paper at ICLR 2022
Regret
Method
5 = 0.5
= 0.7
= 0.9
0 = 0.95
= 0.99
Uniform
100.00 = 0.08
100.00 = 0.09
100.00 125
100.00 0.77
100.00 0.77
Neural Linear
0.95 + 0.02
1.60 + 0.00
4.65 2018
9.56 0.36
49.63 2.41
MAML
2.95 0.12
3.11 0.16
4.84 0.22
7.01 0.33
22.93
Cumulative
NP
1.60 +
1.75 +
3.31 0.100
5.71
22.13
ANP
2.17 # 1.89
3.59 + 8.0
5.63 8.48
11.68 8.97
24.75 #: 7.08
BANP
2.04 + 1.52
2.34 + 1:23
4.30 0.77
6.76 + 1.08
21.18 # 1.09
NeuBANP
0.85
1.02
1.85 + 0.56
3.04 + 0.88
9.76
Uniform
100.00 0:45
100.00 0.78
100.00 +: 1.18
100.00 +: 2.21
100,00 1:4.1
Neural Linear
0.33 0.04
0.79 0:07
2.17 0.14
4.08 0.30
35.89
MAML
2.49 +
3.00 + 0.35
4.75
7.10 0.77
22.89 =
Simple
1.04 + 0.06
1.26 + 021
2.90 2.35
0:447
21.45 +
ANP
0.99 4: 1.68
1.50 +: 2.11
3.64 4:4.1
6.32 7:33
21.65 #: 1.72
BANP
1.22 1.83
2.37 4: 3.04
3.27
4:533
7.73
12.16
20.63
34.11
NeuBANP
0.86
1.04
1.88
3.09
0.23
9.96
Table 1. Results of the wheel bandit problem according to the value of 6. Mean and standard deviation for cumulative regret and simple regret 50 reported. Regrets normalized to that of the uniform policy.
and fourth columns show the explored points by BANP and NeuBANP, respectively. most cases, NeuBANP converged to the optimum faster than BANP. The third and fifth column shows the contour plots of acquisition functions of each method. Note that NeuBANP accurately infers the potential area of the optimum compared to BANP. As explained above, NeuBANP can estimate heterogeneous uncertainty. and thus it is able to handle well for the exploration-exploitatioo trade-off, which is essential sequential prcision-making problems.
4.3 CONTEXTUAL MULTI-ARMED BANDIT
We conducted CMAB experiment, the wheel bandit problem, as in Garnelo et al. (2018b) to show that NeuBANP works efficiently as well as BO based on performance of uncertainty estimation.
Settings We followed the same environment in Gareelo et al. (2018b), but used UCB policy. For more details, please refer to Appendix B.4. The parameter determines the environment of the wheel bandit problem. As 8 increases, high-reward observation becomes sparse, which makes the problem more difficult. We set the baselines of CMAB experiment to MAML (Finn et al., 2017), Neural Linear (Riquelme et al., 2018) and NP. We measured cumulative regrets and simple regrets for 2,000 iterations to demonstrate the performance of each model.
Results NeuBANP performed well for various values (see Table 1). Note that NeuBANP showed better performance in challenging environments with sparse high rewards. NeuBANP showed the ability to learn various reward distributions based on appropriate functional uncertainty modeling. The result demonstrates that ANU can utilize a small number of context information and make accurate estimations. NeuBANP also has stable results its small variance shows. On the other hand, ANP has extremely high variance in the performance because it overfits to certain prediction and failed to adapt to the various bandit environments.
RELATED WORK
Neural Processes CNP (Garnelo et al., 2018a) uses pair of encoder and decoder networks to produce predictive posterior distribution over functions. NP (Garnelo et al., 2018b) introduces global latent variable to embed functional uncertainty the deterministic architecture of CNP and predicts various outputs given the same context data. ANP (Kim et al., 2018) then enhances predictive accuracy by replacing MLP modules in NP with the attention modules. BNP (Lee et al., 2020) proposes the bootstrap method to model uncertainty in stochastic processes without the assumption of single latent variable on which previous methods rely. In addition to these works, there are many attempts use NPS various tasks. Singh et al. (2019) tackles sequential stochastic processes, where the
8

---

Under review as a conference paper at ICLR 2022
dynamics of the given system changes the time being. Leveraging time-variant context points, Singh et al. (2019) models the underlying temporal 3D structures. Gordon et al. (2020) extends NP families to contain translation equivariant functions, providing theoretical formulation to represent wranslation-invarii
functional representations. Louizos et al. (2019) do not assume explicit glohal latent variables, instead supported by dependency graph among local latent variables, to encode inductive bias for given data casier than NPS that use global latent variables.
Bootstrapping Neural Networks Bootstrap method (Efron, 1987) is a reliable approach to esti- mate predictive uncertainty (Lakshminarayanan et al., 2017; Osband et al., 2016). However, it is computationally inefficient to go through the feed-forward computation as much as the number of bootstraps; hence. it discourages the practical application of bootstrap in neural networks. There have been several works to circumvent this issue by approximating bootstrapped distribution. Amortized bootstrap (Nalisnick & Smyth, 2017) approximates bootstrap distribution over model parameters by using amortized inference and implicit models. Generative Bootstrap Sampler (Shin et al., 2020) proposes computational bootstrap procedure that constructs generator function bootstrap evalu- ations for classical statistical models. Neural Bootstrapper (Shin et al., 2021) suggests a simple recipe for generating bootstrapped predictive distributions of MLPs and convolutional neural networks.
Meta-Learning based Stochastic Optimization NPs trained with meta-leaminn framework to solve various tasks related to the data generation process through a single optimization. Santoro al. (2016); Chen et al. (2017) follow the same training procedure of NP. However, Santoro al. (2016) proposes an memory-augmenter network for robust meta-learning, while Chen et al. (2017) 7) proposes the method produce algorithm for black-box optimization using recurrent networks. Sharaf & Daumé III (2019) presents a meta-learning algorithm for learning a good exploration policy the contextual bandit. Ravi & Beatson (2019) also solves contextual bandits based on the Bayesian framework by inferring a posterior on weights of neural networks. Galashov et al. (2019) introduces a unified framework for applying NP wide range of sequential reeision-makingg problems.
CONCLUSION
We have proposed NeuBANP, novel bootstrap method for family of NP model functional uncertainty appropriately. Instead of the standard bootstrap, NeuBANP learns to construct a generator function that produces valid bootstrapped distribution without resampling which be considered as a learn-to-bootstrap method. NeuBANP successfully worked in a meta-learning framework, providing diverse trajectories of underlying data-generating processes, consistent any given context. In addition, NeuBANP estimates the local uncertainty accurately, resolving overfitted prediction and variance overestimation problems observed in both ANP and BANP. We replace the additional layer and repetitive computations in BANP with the simple attachment of bootstrap weights to the model, which leads to lower computations and smaller memory. NeuBANP shows superior performance to previous NP methods in regression and stochastic optimization tasks, including multi-dimensional setting, which has been the desired application of NP. However, due to the numerical instability GP, there is limit in sampling high- nigh-dimensional stochastic processes for the meta-learning framework. We suggest that this is primary task for scalable applications of NP in stochastic optimization, as challenging research direction.
REPRODUCIBILITY
For reproducibility of experimental results, we provide a link to anonymous github that contains our source code Appendix B. The source code includes the implementation of our model, data generation, and experiments. Additionally, the data generation steps are thoroughly explained in Appendix B.
REFERENCES
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016.

---

Under review as a conference paper at ICLR 2022
Maximilian Balandat, Brian Karrer, Daniel R. Jiang, Samuel Daulton, Benjamin Letham, Andrew Gor- don Wilson, and Eytan Bakshy. Botorch: framework for efficient Monte-Carlo bayesian opti- mization. In Advances in Neural Information Processing Systems, 2020.
Yutian Chen, Matthew W. Hoffman, Sergio Gómez Colmenarejo, Misha Denil, Timothy P. Lillicrap. Matt Botvinick, and Nando de Freitas. Learning to learn without gradient descent by gradient descent. In International Conference Machine Learning, pp. 748- PMLR, 2017.
Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. Emnist: Extending MNIST to handwritten letters. In International Joint Conference on Neural Networks, pp. 2921-2926. IEEE, 2017.
Bradley Efron. Better bootstrap confidence intervals. Journal of the American statistical Association, 82(397) 1987.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leariing for fast adaptation of deep networks. In International Conference Machine Learning, pp. 1126-1135. PMLR, 2017.
Alexandre Galashov, Jonathan Schwarz, Hyunjik Kim, Marta Garelo,, David Saxton, Pushmeet Kohli, S.M. Ali Eslami, and Yee Whye Teh. Meta-learning surrogate models for sequential decision making. arXiv preprint arXiv: 1907, 2019.
Jacob Gardner, Geoff Pleiss, Kilian Q Weinberger, David Bindel, and Andrew Gordon Wilson Gpytorch: Blackbox matrix-matrix gaussian process inference with gpu acceleration. In Advances in Neural Information Processing Systems, volume 31, pp. 7576-7586, 2018.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S.M. Ali Eslami. Conditional neural processes. In International Conference on Machine Learning, pp. 1704-1713, PMLR, 2018a.
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J Rezende, SM Eslami, and Yee Whye Teh. Neural processes. arXiv preprint
107.01662, 2018b.
Jonathan Gordon, Wessel P. Bruinsma, Andrew Y. K. Foong, James Requeima, Yann Dubois, and Richard E. Turner. Convolutional conditional neural processes, 2020.
Hyunjik Kim, Andriy Mnih, Jonathan Schwarz, Marta Garnelo, Ali Eslami, Dan Rosenbaum, Oriol Vinyals, and Yee Whye Teh. Attentive neural processes. In International Conference on Learning Representations, 2018.
Jungtaek Kim. Benchmark functions for bayesian optimization. https:///ithhb.coe
jungtaekk ittiii//abba htm//aayeoobbeeeeeee rks, 2020.
Jungtaek Kim and Seungjin Choi. BayesO: Bayesian optimization framework in Python. https:
//bayeso. , org, 2017.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (Poster), 2015.
Balaji Lakshminarayaman, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, 2017.
Tuan Anh Le, Hyumjik Kim, Marta Garnelo, Dan Rosenbaum, Jonathan Schwarz, and Yee Whye Teh. Empirical evaluation of neural process objectives. In Advances in Neural Information Processing Systems Workshop on Bayesian Deep Learning, 2018.
Juho Lee, Yoonho Lee, Jungtaek Kim, Eunho Yang. Sung Ju Hwang, and Yee Whye Teh. Bootstrap- ping neural processes. In Advances in Neural Information Processing Systems, 2020.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In International Conference on Computer Vision, pp. 3730-3738, 2015.
Christos Louizos, Xiahan Shi, Klamer Schutte, and Max Welling. The functional neural process. arXiv preprint arXiv: 19990000322 2019.
10

---

Under review as a conference paper at ICLR 2022
Eric Nalisnick and Padhraic Smyth. The amortized bootstrap. In International Conference Machine Learning 2017 Workshop on Implicit Models, 2017.
Michael A. Newton and Adrian E. Raftery. Approximate Bayesian inference with the weighted likelihood bootstrap. Journal of the Royal Statistical Society: Series B (Methodological), 56(1): 3-26, 1994.
lan Oshand, Charles Blundell, Alexander Pritzel, and Benjamin Van Roy. Deep exploration via bootstrapped DQN. arXiv preprint arXiv:1602.04621, 2016.
Jens Præstgaard and Jon A Wellner. Exchangeably weighted bootstraps of the general empirical process. The Annals of Probability, pp. 2053-2086, 1993.
Carl Edward Rasmussen. Gaussian processes in machine learning. In Summer School on Machine Learning, pp. 63-71. Springer, 2003.
Sachin Ravi and Alex Beatson. Amortized bayesian meta-lcarning, In International Conference on Learning Representations, 2019.
Carlos Riquelme, George Tucker, and Jasper Snoek. Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In International Conference on Learning Representations, 2018.
Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta- learning with memory-augmented neural networks. In International Conference on Machine Learning, pp. 1842-1850. PMLR, 2016.
Amr Sharaf and Hal Daumé III. Meta-learning for contextual bandit exploration. arXiv preprint 1XXiv:9901008130 2019.
Minsuk Shin, Lu Wang, and Jun $ Liu. Scalable uncertainty quantification via generativebootstrap sampler. arXiv preprint arXiv:2006.000 200660077. 2020.
Minsuk Shin, Hyungjoo Cho, Hyun-seok Min, and Sungbin Lim. Neural bootstrapper. In Advances in Neural Information Processing Systems, 2021.
Gautam Singh, Jaesik Yoon, Youngsung Son, and Sungjin Ahn. Sequential neural processes. In Advances in Neural Information Processing Systems, volume 32, 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
A MODEL
A. BASIC OPERATION
Muti-Layer Perceptron MLP(d;, dr, (o, denotes the multi-layer perceptron consisting of ny
linear transformations and ReLU activations between them. Let parameters d,, da, d, the dimension of input, hidden and output feature, repectively. We used same d, for every hidden layers. Lin 7) denotes the linear transformation of input with feature dimension p into output with feature dimension
(LP da (X) Lin(da, do) (ReLU Lin(da, d=2) ReLU Lin(d,, (X)
(12)
Dot Product Attention DotProdAttn (Vaswani et al., 2017) denotes the attention operation with attention score based on cosine similarity. Dot product query (Q) and key (K) tensors calculates the similarity of each vectors in query tensor relative each vectors in key tensor. Attention score is calculated through softmax operation of normalized similiarity. Let de the feature dimension of key and query tensors, and d, that of value tensors.
DotProdAttn( Q, K, V) softmax (Q" K/ K/VdaVV
(13)
11

---

Under review as a conference paper at ICLR 2022
Multi-Head Attention MHA denotes multi-head attention with dot product attention. The input is pre processed with MLP and the output of attention is post-processed with layer normalization. For simplicity, omit parameters of each MLP layers. split(X,, n) denotes splitting of tensor X with respect to feature axis into a tuple of n tensors (x))== which have same dimension in feature axis. [X1, X2, Xn. denotes the concatenation of tensors X1, X2,,-., X,, with respect feature axis. LayerNorm denotes the layer normalization introduced in Ba et al. (2016). Let Ahead the number of heads in multi-head attention.
(Q)) split( ML.P, 1y.. (Q), Rhead)
(14)
(K)) split (MLP q/: (K), (hhead)
(15)
(V.) Rhaad split(MLL,, (V), 1hhaad)
(16)
MHA(Q, V) = LayyeNorm((DDoorootttt Kg (m)
(17)
We used self-attention to calculate efficient representations of context De (Xe. Yc). We define self afffnattntion based on multi-head attention as follows:
Self-Attention
D' (ee Yc)
([ Vel::Vo)ecc. SelfAun(De) = MHA D()
(18)
Cross-Attention We used cross-attention to calculate representation of context specific to target feature x in interest, when original representations (he)eec is given. We define cross-attention based multi-head attention follows:
CrossAtt(Er (bo)oEc) == MHA Xc, ((he)oEc)).
(19)
A.2 ARCHITECTUREE
For fair comparison, we used the same architecture of all models as in Lee et al. (2020). For NeuBANP, we increased the input dimension of SelfAutn in the posterior path by one to take bootstrap weight as additional input. Please refer to Lee et al. (2020) for detailed model architecture of ANP and BANP.
A.3 NON-ATTENTIVE CASE
As an ablation study, we consider the non- non-attentive case of NeuBANP, called Neural Bootstrapping Neural Processes (NeuBNP). Like the architecture of NeuBANP based on ANP, the architecture of
this model is based on NP, the non-attentive counterpart of ANP. With ( = { De, we cee as
defined in 3, we applied the similar strategy of using random weights in the encoder follows, and trained with the same loss function:
z = {>(0) JoEc = (L)
(b)
C
( = mcan
2
w
(20)
(b) ( leEe ( (b) mean (
(b)
(b)
he
(21)
(22)
To make the non-attentive counterpart of NeuBANP, we use random weights in both paths, re- sulting in two latent variables 2(6) (b), They induce randomness into the decoder output g( = MLP(z, z()) (6)), and the predictive distribution is construct by (10). The results in Table 2 and Table 3 shows that this model performs worse than NeuBANP, but outperforms BNP and NP, showing
the quality of functional uncertainty the model learns with the bootstrap.
B EXPERIMENTS
Implementation of NPS other than NeuBANP was borrowed from the source code of Lee et al. (2020)². Regression and Bayesian optimization experiment done in single GeForce RTX 2080 77 GPU with the memory of 11, 019 MiB. dulti-dimensiona. regression including image complction was done in Tesla V100 GPU with the memory of 32, 480 MiB.
"https://anonymouu 4open. science////ccccccccccccccccccnce wwtpebnnnnnniiiteic initiall 'https:// thub. com/ juho- Tee bee/bn, MIT License.
12

---

Under review as a conference paper at ICLR 2022
NP
BNP
22
-2
-2.0
-0.5 0.0
0.5
2.0 -2.0
0.5 1.0
2.0
Fig. 5. Each plot shows predictions given by NP, BNP in lincar regression. The ground-truth function is simple linear function with heterogeneous variance: y
3e(x) where e(x) [ (0 (x)) and BANP, considering that overfitting does not occur in NP and BNP, leading cause of overfitting.
(a)) v 10-3, We used the official code provided by Lee et al. (2020). Unlike the case of ANP and
notice that attention modules are the
B. SIMPLE LINEAR REGRESSION
We experimented using linear function rather than a function sampled from the GP to analyze how well the models work in a simple regression task. Additionally, we need to examine how well the NP models predict uncertainty in the presence of heterogeneous noise in the data, and we add c(x) ~ (O,, (z)), (z) vx² + 10-5 to the linear function. We multiplied the noise follow the meta-learning framework, and the value was uniformly randomly sampled from 10.1, 1.0] during training. In this task, NP, BNP, and NeuBANP were able to estimate underlying true linear function and the uncertainty of heterogeneous noise. On the other hand, in case of ANP and BANP, overfitting occurred and failed to predict the true function and its uncertainty (see Figures and 5). As explained the main text, we can see that the overfitting phenomenon occurs because of the attention mechanism that has appeared solve the underfitting issue of the NP. NeuBANP has the advantage the attention mechanism and achieved better performance through regularization using random weights. For (A)NP and BA, used the official code provided by Lee et al. (2020). The remaining training settings are the same as the ID regression in Appendix B.2, and we changed only the training iteration to 10, 000.
by coefficient
B.2 ID REGRESSION
Training For all models, training dataset consists of randomly sampled context and target from functions following GP with RBF kernel k(x, y) = s² exp(-1læ: yll /// Parameters of kernel are randomly sampled with Uniform(0.1) 1.0) and ~ Uniform (0.1, 0.6). Feature values (xi)ECCT is chosen uniformly at random in 2_. The size of context and target are randomly sampled with |C] ~ Uniform(6 (3, 47) and [T| ~ Uniform(3 50 [C]). We trained all models for 100,000 iterations and used Adam optimizer (Kingma & Ba. 2015). For stable learning, we used the cosine annealing scheduler with initial learning rate 14-
Results Table 2 shows ligeliieli of NPS for various evaluation dataset sampled from with RBF, Matérn 5/2, Periodic kernel. As in generation of training dataset, parameters of Matérn 5/2 kernel k(x, y) 22 v5 | yl 2 (1/3()))) exp(-v5||ar - yll/t) and Periodic kernel k(x, y) 2 exp( 2 (T||z l Uniform(0.11 0.6) and p (0.1, 0.5). For RBF and Matérn 5/2 dataset, NeuBANP showed state-of-the-an performance both in fitting context and predicting target. We added figures showing the predictions of ANP, BANP, and NeuBANP for the Matérn 5/2 and Periodic kernel (see Figure 6 and 7). In the case of the Matérn kernel, the two problems described in the main text occurred identically for ANP and BANP (see Section 4.1). However, in the case of Periodic, we can that all models failed to approximate the true function correctly. This result came out because experimented with testing the models' generalization performance when trained with the RBF kernel. And the quantitative results in Table 2 show that the prediction performance of the attention based
/p){() was randomly sampled with ~ Uniform 1.0),
13

---

Under review as a conference paper at ICLR 2022
RBF
Matérn 5/2
Periodic
Method
context
target
context
target
context
target
CNP
1.17 # 0.08
0.87
+ 0.36
1.06
0.11
0.65
+: 0.39
-0.31
+ 0.41
-2.05
L.17
NP
1.11 + 0.09
0.78
+ 1.47
0.99
+ 0.11
0.56
+ 0.50
-0.28
0.37
-1.73
+ L.9
ANP
1.38 0.00
1.08 041
1.38
0.94 0.47
0.21
0.76
-6.82
2.8
BNP
1.20
0.92 0.4
1.09
0.72 0.35
-0.18 037
-1.16
0.56
BANP
1,38 1: 0.00
1.12 1: 0.33
0.00
0,99
1: 0.38
0.28 1:
-5.69
1:37
NeUBNP
1.54 1: 0.20
1.01
+: 0.55
1.22
1: 0.22
0,53 1: 0.59
-0.34
= 0.45
-2.34
+ 1.95
NeuBANP
3.17 0.28
1.38
= 0.60
3.09
0.29
1.13 + 0.4
1,56 = 0.73
-11.49
= 8.08
Table 2. Log-Likelihood NPS for 48,000 different evaluations of context and target.
model the target data is poor. Among them, NeuBANP has the worst performance, and think the reason is the lower bound on the predicted variance set by ANP and BANP. Quantitative results show similar predictions for all models, but ANP and BANP achieve numerically more robust performance by setting lower bound on the variance. We find some cases with the jumps in the function values which do not seem like smooth function (See Figure 3, 6, and 7). This phenomena occurs in every attentive models including ANP, BANP, and NeuBANP. It looks like distorted prediction on particular region, however, since our model predicts high variance such region, this does not raise a problem in predicting the global trend, as we can see in high average log likelihood in 1d regression.
ANP
BANP
NeuBANP
-22
-62-
-0.2
64
64-
04
-02
-02-
-22
-04
-64-
-54
Fig. 6. Comparison of ANP, BANP, and NeuBANP in ID regression. Matérn 5/2 kernel case.
ANP
BANP
NeuBANP
6:0-
-0.
-0.5
-0.5
-1.0
-1.0
-05
-25
-2.
-1.
-10
-100
Fig. 7. Comparison of ANP, BANP, and NeuBANP in 1D regression. Periodic kernel case.
B.3 BAYESIAN OPTIMIZATION
One-dimensional case Table shows the performance GP and various NPs for ID Bayesian optimization task. RBF, Matérn 5/2, and Periodic kernels used generate evaluation dataset.
14

---

Under review as a conference paper at ICLR 2022
Algorithm 1: Neural Process based Bayesian Optimization.
Input Target function f"; Acquisition function U;; Observed data Do = {(00. f (zoo))) Maximum evaluation step T.
Meta-train neural process po on f ~ P(F).
for f ,,...,? do
Find x, by optimizing acquisition function: I, = argmin 4 (pa (1-11)
ZEd
Evaluate (xz) and update the observed data: D, + D,-1 U (1I,, (* (z)))
ABF
Motêm 5/2
Periodic
6.0
5.0
Simple
6.5
6:06
Borations
Rocations
Rorations
Fig. 8. ID Bayesian optimization results. Bold lines show the mean of simple regrets over 100 experiments. We also report 10% of the standard deviation.
Ackley
Dropwave
complexity
Shnpl DOME
Trs
05
04
02-
Ackley
Rostrigin
Time complexity
115
T1me 130
SUPOM 19O0e
130
C55
Serratooss
Latrll CETS
Numbbrr of eralcatedd points
Fig. 9. Left: Multi-dimennsionll Bayesian optimization results various benchmark functions with acquisition function. Bold lines represent the mean performance over 100 experiments. We indicate 20% of standard deviation. Right most: Time complexity of each model as the number of observations increases.
Multi-dfimensionl case In multi-diimensional BO experiments, used GPyTorch (Gardner et al., 2018) for scalable GP regression, and BoTorch (Balandat et al., 2020) for overall Bо process (e.g., optimization of acquisition functions). GP was to the default setting of BoTorch. detail, GP model was parameterized with Matérn 5/2 kernel with ARD and constant mean function, and prior distribution for hyperparameters was set as Gamma(3, 6) for length scale and Gamma(2, 0.15) for output scale For three-dimensional BO experiment in Figure 4, the overall time complexity of ANP and BANP almost the same. This result is seemingly in contrast with the fact that ANP takes a shorter time for prediction than BANP. However, since the BO algorithm contains the optimization of the acquisition function, the qualities affect the overall time complexity. We conjecture that BANP gives acquisition function easier to optimize than that of ANP so that the overall time complexities of both models are similar. Additionally, did not report the results of GP for the two functions. Specificallyy omitted the Goldstein- -Price
acquisition functions obtained by the model predictions may
'https:/ / wwt.bb..c..cocccc.cooooo wwwpreeeelluuuppppeeee wwwpsppppptpop/ptth/hhheop/ MIT License. 'https:/ //gi // thub. htwb..ccmm/pttoccc/bttttttee
MIT License.
15

---

Under review as a conference paper at ICLR 2022
ALkkey
Fig. 10. First column: 2D objective functions Second Fourth columns: Contour plots of functions and the evaluated points during Bayesian optimization. Third & Fifth columns: UCB value the iteration.
Method
RBF
Matérn 5/2
Periodic
GP (RBF)
0.016 0,922
0.048
0.104 :: 0.242
CNP
0.072
0.081 + 0.187 + 0.303
0.096 : 0.083 + 0,121
NP
0.154 0.273
ANP
0,209 0.364
0,223 + 0.328
0.107 + 0.142
BNP
0.109 0.214
0.105
0.188
+ 0.071
BANP
0.114
0.136 0.256
0.077
NeuBNP
0.069 0.169
0.125
038
0.058 :
NcuBANP
0.006 : 0.011
0.011 i: 0,055
0.028 i: 0.035
Table 3. ID Bayesian optimization results. Mean and standard deviations of simple regrets over 100 runs are reported.
since the simple regret value of GP was too large (the performance was poor) compared to other models and omitted the Rastrigin since the 46 errors occurred out 100 experiments. An error may occur when ill-conditioned data is given during the kernel training process of the GP, and the data recommended by the UCB during the exploration (or exploitation) process seems to correspond to this condition. If we report the performance ignoring the numerical error, the simple regret for the Goldstein- Price was 52049.13, 13, and the simple regret for the Rastrigin was 18.48. For the Rastrigin function, GP with UCB is numerically unstable, but like EI, achieved the best performance.
B.4 CONTEXTUAL. MULTI-ARMED BANDIT
Setting We followed wheel bandit setting introduced in Riquelme et al. (2018). At every step {< T), wo-diimensioaal point (Iz, y1) inside the unit circle Runit {(x,)) I² x2 +y < is given as context c, The algorithm chooses an action o, E {1, 2, 4, 5). The stochastic reward r: r(a,, 0t) is sampled from the reward distribution. Let R1, R2, Ra. R4, R3 Runte the disjoint
16

---

Under review as a conference paper at ICLR 2022
Dim
Target
GP
ANP
BNP
BANP
NeuBANP
Ackley
2.84 +
0.19 = 0.53
1.82
0,50 1.03
0.08 0.27
Dropwave
0.36 + 0.17
0.22 = 0.15
0.32 + 0.18
0.28 0
0.15 +: 0.10
2D
Goldsteinprice
475.52 469.5
2098.22 + 150988
80.67 + 66.85
30.33 : 25.42
Michalewicz
0.67 0.45
0.61 = 0.40
1.00 0.46
0.69 0.38
0.45
Ackley
3.39 1.25
5,36 =: 0,97
3.29 1.11
4,30 1.15
0.34 0.26
Cosine
0.04 0.44
0.10 0.10
1.12 0:57
0.25
0.005 0.003
3D
Hartmann
0.42 0.77
0.33
1.94 0.6
0.93
0.39 =
Rastrigin
54.94 = 19.84
48.55 # 14.34
38.36 + 8.20
23.06 # 19.77
Table 4. dittesiiensii Bayesian optimization results. Mean and standard deviations of simple regrets over 100 runs are reported.
sets (regions) of unit circle as follows:
R1 {(x, <
(23)
R₂
0,yx00
(24)
R3 ( <0,,0
(25)
R4 ((.,)::0<"=="<<x<00)c
(26)
R₂ ((x,)) : ,, 0,,>0,y<< 0,
(27)
where the constant determines the size of R₁ relative to other regions. Each action results rewards following different distribution according to the region to which the given context belongs, where N denotes the normal distribution.
r(1,c) ~ NV..2,
(28)
((55,0.011) if R. Va E (2,3,4.5)
r(a,c) ((1,0.)19) otherwise
(29)
Note that action {1} always produces moderate reward, but the other actions {2, 5, sometimes produce very high reward when the context is sampled from the corresponding high-reward region. Thus, learning different reward distributions for actions {2, 3, 4, 5} by apprehension of context information is critical to bandit performance. As 5 increases, the high-reward regions for each actions become smaller. Then the model should learn from rare observation of such high reward, which means the problem becomes more difficult.
When pre-training NeuBANP, as in Garnelo et al. (2018b), 8 training batches 512 contexts and 50 targets were generated from the environment with hyperparameter randomly sampled; 6 ~ Uniform(O, 1). We consider two-dimensional context point c; as feature I; E R2 and five rewards for actions (r(1, c)), r(2, r(,, c,), r(4, r(,, cz)) as label y E R At evaluation, only rewards for chosen actions observed by the model. Thus, replace other unobserved rewards with dummy values randomly sampled from (0,, 1), following the usual strategy.
Training and Evaluation
B.5 IMAGE COMPLETION
Settings We compared the baseline NPs and NeuBANP on image completion tasks. Following Lee et al. (2020), we trained all models on EMNIST (Cohen et al., 2017) and CelebA (Liu et al., 2015) which was resized to 32 x 32. For EMNIST, we used only 10 classes for training and reported the evaluation results for both seen classes and unseen classes separately. NeuBANP was trained with 10 samples, and the other baselines were trained with 4 samples. For evaluation, we used 50 samples for all methods. Similar to ID regression experiment, we randomly select the pixels of a given image as context/target, and the number of context/target were drawn randomly from Uniform distribution. However, in this case, we increased the maximum number of given points; ie., |C] ~ Uniform 197), [T| ~ Uniform(3, 200 - Cl). x values were rescaled to [-1, and the corresponding y values were rescaled to [-0.5, 0.5]. We trained all models for 200 epochs and set a initial learning rate of 5 x 10 using the Adam optimizer (Kingma & Ba, 2015) with cosine annealing scheduler for learning rate decay.
17

---

Under review as a conference paper at ICLR 2022
Orielied
MAY 2
3
3
3
3
Seen
context
Inrget
context
farget
CNP
0.926 200
0.751 c0e
0.766 =oom
0.498 1112
k
NP
0.948
(1.80
0.808
0.600)
1.383
0.993
1.383
0.894
1.004
0.880
0.883
0.722
RANP
1.383 2000
1,010
1.382 -000
mm
NEURANY
1.476 2.
1.387 g1003
1.333 g001
1.119 g0.0
Fig. 11. Qualitative result of EMNIST image comple- tion.
Table 5. Quantitative result of EMNIST image comple- tion. Mean and standard deviationd of likelihonod over experiments.
Occptnal
copcest
APP
DASY
context
target
CNP
2.975 0013
2.199 1: 0.003
NP
3,066 0011
2.492 1 0.014
ANP
4.150 + 0.000
2.731
+ 0.006
BNP
3.269
+ 3.129
2.788
BANP
4. 149
+ 0.000
+ 0.005
NCUBANP
13.946
2.870
Fig. 12. Qualitative result of CelebA image comple- tion.
Table 6. Quantitative result of CelebA results. Mcan and standard deviationd of likelihoood over 5 experiments.
Results Figure 11 and 12 show the mean prediction and uncertainty estimation of ANP, BANP, and NeuBANP for test images in unseen classes. For both datasets, though our model shows noisy mean prediction due to the random weights, we can demonstrate the advantage of NeuBANP in uncertainty estimation. NeuBANP estimated the uncertainty correctly in the area where the color of the pixel changes and thus possesses significant uncertainty. This leads to the overall improvement in quantitative performance (see Table 5 and 6).
B.6
TIME COMPLEXITY
Settings We measured the time complexity empirically according to the number of context points, target points, and bootstrap samples. We fixed the number of targets 25 and adjusted the number of contexts to 10, 20, 30, 40, and 50 to see how inference time varies with the number of contexts. Conversely, to see the inference time according to the number of targets, we fixed the number of context points to 25. We fixed the number of bootstrap samples to 50 as in the 1D regression experiment. When conducting experiments with the number of bootstrap samples, the number of context and target points was fixed at 20 and 25. All experiments were conducted with batch containing 100 tasks.
BANP places remarkably high computational cost in that the approach of bootstrapping the attention module is inefficient, as demonstrated in Figure 13. The inference time becomes noticeably longer as the number of context points increases. NeuBANP, on the other hand, learns to bootstrap efficiently: therefore, its time complexity is comparable to that of BNP, which does not use the attention module.
Results
Number of contexts
Number of targets
Number of bootstrap samples
Method
10
20
30
40
10
30
40
10
100
1.797
1.977
2.222
2.546
2.886
1,830
1.882
1.965
2.057
2.156
1.532
1.639
1.950
BANP
3.512
4.405
5.345
6.509
7.793
4.439
4.626
4.834
4.926
5.117
3.189
3.699
4.775
NeHRANP
1.632
2.217
2.757
1.768
2.063
2.212
2.357
1.606
Table 7. Inference time measurement. Mean of inference time over runs are reported.
18

---

Under review as a conference paper at ICLR 2022
8-
NeuBANP
4.5
BNP
4.5
4.0
3.0
3.0
2.5
2.0
15
10
20
30
10
20
30
40
50
10
100
Number
Number targets
Number bontstrap samples
Fig. 13, Inference time measurement.
19


Reviews:
[{"review_id": "bCFeQifUzWr", "reviewer": "Reviewer_B7VB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The strength of the paper is that the authors report multiple experiments in different settings that show the superior performance of NeuBANP. They are clear about the comparative advantage from ANP, BANP, and NeuBANP. The figure comparing BANP and NeuBANP makes it easy to understand the difference and highlights the efficiency of NeuBANP. The figures for experimental results also depict the superior performance of NeuBANP well.\n\nMy main concern to hold this paper to the high standard of ICLR is the significance and originality of the paper. As it has been discussed well in the paper, Bootstrapping Attentive Neural Processes have been proposed in previous work. Also, Neural Bootstrapper, which replaces the sampling problem of bootstrapping with the augmentation of the loss function, has also been proposed in Shin et al. 2021. This paper looks to me is an application of the ideas in (Shin et al 2021) to the published Bootstrapping Attentive Neural Processes. The paper is well written and the ideas are discussed clearly. However, I am not sure if a combination of existing ideas can win out in a highly competitive venue such as ICLR.\n\nThere are some suggestions for the paper as well. First, it would be helpful to mention the significance of NP models compared to other models a little more. Second, one of the contributions the authors mention is computational efficiency. Although this is well depicted in the figure, some more experimental results regarding this point would be better for highlighting this point. Third, if there exist other NP methods apart from ANP and BANP, it might be good to introduce them briefly. Fourth, it would be better if the others made it clearer if the state-of-the-art performance is regarding all other machine learning models or other NP methods. Lastly, including some mathematical details or reasoning of why NeuBANP does better in estimating functional uncertainty would be good. \n", "summary_of_the_paper": "The authors propose a new bootstrapping method in NP family called NeuBANP. The authors acknowledge the limitations of BANP and used the neural bootstrapping method to make development from BANP. While BANP does better functional uncertainty estimation than ANP, it carries a higher computational burden compared to ANP since it requires multiple computations of the encoder network, the adaptation layer, and additional heuristics. By incorporating neural bootstrapping to the bootstrapping procedure, NeuBANP is a more computationally efficient model, estimates functional uncertainty better than BANP, and also alleviates the overfitting problem that ANP and BANP carries. Experimental results show that it achieves state-of-the-art performance on stochastic optimization problems, including multidimensional Bayesian optimization and contextual multi-armed bandit.", "summary_of_the_review": "The authors propose a new bootstrapping method in NP family called NeuBANP. The authors acknowledge the limitations of BANP and used the neural bootstrapping method to make development from BANP. The experimental results show great improvement over other NP methods. While the paper is written well and the figures are clear, there is some room for improvement. In particular, the combination of existing ideas may not hold up to the high standard of ICLR.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "YqPiQ6AOu0x", "reviewer": "Reviewer_Me1R", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n- The paper is written in clear manner.\n- The proposed method is easy-to-implement in NP schemes.\n- The code is also provided together for the reproducibility.\n\nWeakness:\n- The novelty is somewhat limited, that this is a direct utilization of Neural Bootstrapper.\n- I'm confused on the message of the paper. In Fig 1, the authors argue that the baselines (1) overfits the data from the linear regression samples, and (2) are incapable of capturing uncertainty. However, in Fig 3, the proposed NeuBANP seems overfitted and the baselines seems that they can capture the uncertainty (at least) in interpolation. If I missed something, please let me know.\n\n\n", "summary_of_the_paper": "The paper proposes Bootstrapping Attentive Neural Processes (NeuBANP), which utilize random sum-to-one weight in the encoder following from Neural Bootstrapper (Shin et al., 2021). The authors argue that this utilization resolves the overfitting problem in attentive NPs, and the proposed NeuBANP is more efficient in computation and memory perspective. Various experiments, such as synthetic examples and contextual multi-armed bandit, are conducted to compare against previous works of NPs and GP.", "summary_of_the_review": "My opinion is that the paper is on borderline. The work seems neat and probable while the novelty is limited. Considering the quality of the venue, I'm bit negative on the direct utilization of existing work. Also, there is a disalignment (probably my misunderstanding) of the motivation and the experiment result, which I discussed in the weakness, want to be addressed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EqkX4uZXKJ4", "reviewer": "Reviewer_1P97", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\n\n- Problem of modeling functional uncertainty is clearly motivated. \n- Simplifies and improves BANP using NeuBoots.\n- Evaluates on standard benchmarks and shows superior results on Bayesian optimization and contextual multi-armed bandit tasks. \n- Provides clean structured code for reproducibility. \n\nWeakness:\n\n- The empirical evaluation on 1D regression remains mixed. It’s hard to tell whether the failure of modeling GP target samples from the periodic kernel (Table 2) is intrinsic to the proposed method or if the experiment is not executed properly without having further analysis. Also it seems like context LL is significantly better for regression tasks but that has not been explained enough. \n\n- One of slight worry from the reviewer is that the paper’s method is hinging on two of very recently proposed approaches (Bootstrapping Attentive Neural Process and Neural Bootstrapper) where each of the merits are not quite validated thoroughly in the literature yet. \n\n\nSuggestions:\n\n- Main interesting idea this paper brings into NP space is utilizing Neural Bootstrapper. However, current analysis heavily relies on building on BANP. It would be great ablation analysis to apply neural bootstrapper on non-attentive models and compare to vanilla BNP and NPs.  While the idea of using attentive neural processes may fade-away,  the idea of efficient bootstrapping of neural processes could remain important. \n\n- Figure 1 / Figure 6 Training setting is unclear and makes it hard to interpret what message is being delivered.\n\n- In section 4.1. 1D regression seems cherry-picked since it only shows the RBF kernel and does not discuss failure in the Periodic (in Appendix) while standard 1D evaluation runs on various kernels (RBF, Matern, Periodic).\n\n- Section 4.3 results on CMAB are quite promising and demonstrate competitiveness. It would be great to include a comparison to BANP and ANP as well to see that Neural Bootstrapping is the main reason for superior performance. \n\n- In Figure 5, it would be interesting to have a sense of order of evaluated points to see how each model zooms into particular (often suboptimal) minima. \n\nQuestions:\n- As mentioned earlier, do the authors have a good idea why improvements on the context point are huge? According to the motivation NeuBANP was introduced to solve ANPs “overfitting” issue. Having high context LLs doesn't quite align with the motivation?\n\n- In Figure 3, NeuBANP samples as well as posterior distribution are discontinuous; why is this the case? (e.g around -0.5, and 0.5) With the discontinuity, it does not seem like a great model but still able to obtain good context/target LLs?\n\n- In Table 5, ANP’s context LL for both seen classes and unseen classes is exactly the same, did authors check that if there’s mistake on reporting or the actual numbers turned out to have the same mean and std. \n\n- While authors cite Lakshminarayanan et al. 2017 for bootstrapping to be a reliable approach to estimate predictive uncertainty in neural networks, the reviewers take from that paper is that bootstrapping was detrimental compared to deep ensemble. In Nixon et al., 2020 (https://openreview.net/forum?id=dTCir0ceyv0), they further observe bootstrapping doesn't improve uncertainty beyond the vanilla ensemble method. Could authors elucidate in what situation we expect using bootstrapping to be beneficial vs not in neural networks?  \n\n- Caption in Table2: is reporting on the 48,000 evaluation standard procedure? seems quite arbitrary and high\n\n\n------------------------------------------------------------------------------\n\npost rebuttal : I thank the author for the response and clarification. My assessment still is that this is somewhat borderline paper for ICLR and slightly leaning supporting acceptance thanks to solid result on sequential decision making tasks. ", "summary_of_the_paper": "The paper proposed a new class of neural process algorithm called neural bootstrapping attention for neural processes (NeuBANP). This method utilizes efficient Neural Bootstrapping (NeuBoots) to improve Bootstrapped Attentive Neural Processes (BANP) in capturing functional uncertainty. Authors show that NeuBANP achieves state-of-the-art performance in benchmark experiments including Bayesian optimization and contextual multi-armed bandits. \n", "summary_of_the_review": "This paper proposes an efficient way of modeling functional uncertainty building on recent work of NeuBoots and BANP. On the benchmark of sequential decision making, authors demonstrated state-of-the art performance which shows great promise. Method is presented in a clear fashion and evaluation is done on standard set up. There are some desired analyzes pointed out in the main review that could improve scientific understanding of the proposed method. Overall, I believe the proposed idea is sound and interesting to be shared among the ICLR audience. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "4_TIuEfvor", "reviewer": "Reviewer_fWXh", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Positive aspects: \n* Overall the idea of extending the bootstrapping element of B(A)NPs using neural bootstrapping makes sense and seems to work well in practice. \n* The authors do provide a nice set of experiments with the appropriate baselines and the results seem promising.\n\n\nCriticisms:\n* My main concern is that it is not clear to me why one would expect that applying this neural version of bootstrapping would improve the uncertainty prediction of the model and allow for the heteroscedastic uncertainty prediction. From the results it seems that this is indeed the case, but it would be useful to have some intuition behind this explained in the paper. The way I understand the paper this is the main benefit of this model upgrade so this should be elaborated on. \n* The second benefit of this model seems to be that it is computationally more efficient (given that it does not require the iterative bootstrapping step, this makes sense). It might be worth analysing how much this buys you in terms of computation/time. Given that this is one of the two selling points it would be nice to have a more quantitative evaluation of this.\n* Finally, in terms of novelty, while this is the first time I hear of this particular combination of models, it is a somewhat straightforward combination of previously existing ideas (this in itself should not render the work unworthy of being published, of course).\n\nSuggestions:\n* I personally find the notation used to state that you are sampling the context points  $C$ from some set of indices $[n]$ a bit confusing. In equation 1 the inner expectation is wrt C but then the conditional probability is on $D_C$. I would personally write it as: $E_{C \\sim D_C}-\\log p_{\\theta}(Y|X, C)$.\n* In the paragraph on BANP I would define the variable $b$ much earlier (right when it is first used in the fourth line) rather than waiting to the end, as it is not clear that these are the different ensemble samples. \n* In figure 2 I would make clear that both the bootstrapping path in B(A)NP and the encoding part in NeuBANP is repeated b times (if I understood correctly this is only shown for one instance in the figure).\n* Also in figure 2 I would add the $\\hat{r}$ variable in the diagram just before the decoder, since you mention it in the text.\n* In figure 3 I would explicitly mention that the minimum variance of ANP and BANP is as large as it appears because that is the fixed minimum value. Not to say that there is no benefit from NeuBANP not having such a lower bound, but for those that might not be familiar with that fixed value in the other models the results might seem odd.\n", "summary_of_the_paper": "The authors introduce NeuBANP, an extension of B(A)NP that replaces the iterative prediction method of B(A)NP with a single prediction step following the idea behind the neural bootstrapper. This new model is computationally more efficient and able to produce more accurate uncertainty estimates as well as correctly model heteroscedastic uncertainties. The authors provide experimental results on nonparametric regression tasks, Bayesian optimisation, contextual multi-armed bandit tasks and image inpainting (in the appendix).", "summary_of_the_review": "Overall I think the paper is nice, the explanations are not hard to follow and it is overall well written. While the novelty of this model is not outstanding, the results section covers an interesting range of experiments with the right baselines. I would personally recommend adding more intuition as to why this extension improves the performance the way it does, as it is the main selling point. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: Overall, the work is borderline with no reviewer feeling strongly for or against the paper.

The paper is well-written and proposes a simple approach, along with code for reproducibility. Criticism stems primarily in the work's technical novelty, being an incremental improvement of ideas from ANP and BANP, and related work like Neural Bootstrapper. In addition, the experimental validation involves regression on 1-to-2D functions, Bayesopt on synthetic functions, and contextual bandits on the synthetic wheel bandit problem. This is fairly toy, and multiple reviewers raise unaddressed concerns on the regression experiments. Ignoring orginality in and of itself (which is overvalued in conferences), the work does not yet provide a sufficiently convincing demonstration of its practical importance.

I recommend the authors use the reviewers' feedback to enhance their preprint should they aim to submit to a later venue.
Decision: Reject