Dropout Enhanced Bilevel Training
Authors: Peiran Yu
Junyi Li
Heng Huang
Venue: ICLR 2024 spotlight (2024)
Keywords: Bilevel Optimization
Overfitting
Primary area: optimization

Abstract: Bilevel optimization problems appear in many widely used machine learning tasks. Bilevel optimization models are sensitive to small changes, and bilevel training tasks typically involve limited datasets. Therefore, overfitting is a common challenge in bilevel training tasks. This paper considers the use of dropout to address this problem. We propose a bilevel optimization model that depends on the distribution of dropout masks. We investigate how the dropout rate affects the hypergradient of this model. We propose a dropout bilevel method to solve the dropout bilevel optimization model. Subsequently, we analyze the resulting dropout bilevel method from an optimization perspective. Analyzing the optimization properties of methods with dropout is essential because it provides convergence guarantees for methods using dropout. However, there has been limited investigation in this research direction. We provide the complexity of the resulting dropout bilevel method in terms of reaching an $\epsilon$ stationary point of the proposed stochastic bilevel model. Empirically, we demonstrate that overfitting occurs in data cleaning problems, and the method proposed in this work mitigates this issue.

Published as a conference paper at ICLR 2024
DROPOUT METHODS FOR BILEVEL TRAINING TASK
Peiran Yu
Junyi Li
Department of Computer Science University of Maryland College Park, MD 20740, USA {pyu12 2 (al122330mmmm uumddmd.ee edu
Department of Computer Science University of Maryland
College Park, MD 20740, USA {junyili ai gma ww..a.l..oo com
Heng Huang
Department of Computer Science University of Maryland College Park, MD 20740, USA [henghuanghh]er waw.r/ghhhhhmmmm....ooooo
ABSTRACT
Bilevel optimization problems appear in many widely used machine learning tasks. Bilevel optimization models are sensitive to small changes, and bilevel training tasks typically involve limited datasets. Therefore, overfitting is com- mon challenge bilevel training tasks. This paper considers the use of dropout to address this problem. We propose bilevel optimization model that depends on the distribution of dropout masks. We investigate how the dropout rate af- fects the hypergradient of this model. We propose dropout bilevel method to solve the dropout bilevel optimization model. Subsequently, we analyze the re- sulting dropout bilevel method from optimization perspective. Analyzing the optimization properties methods with dropout essential because it provides convergence guarantees for methods using dropout. However, there has been lim- ited investigation in this research direction. We provide the complexity of the resulting dropout bilevel method in terms of reaching an stationary point of the proposed stochastic bilevel model. Empirically, we demonstrate that overfitting occurs in data cleaning and meta- meta-learning, and the method proposed this work mitigates this issue.
INTRODUCTION
Bilevel optimization appear in many problems widely used in machine learning tasks such as data cleaning (Shaban al. 2019), hyperparameter optimization, meta-learning (Franceschi al., 2018) and reinforcement learning YYang et al.) 2019). The bilevel optimization problem involves two minimization problems that are stacked on top of each other, where the solution to one optimization problem depends on the solution of the other. Take data cleaning an example. Suppose we have corrupted training data set Der with N₂r data points and clean data set Deat with Neat data points. Let f(w; {) be network parametrized by w. The aim of data cleaning is train the model with corrupted data by solving
Neat Σ
min XER
(((((())))):)) s.t.w(A) E Arg min
N Σ (()))(((
(1)
Noat
where 4) E Draat & E Der. I the loss function and (A:) is the weight of the loss w.r.t. the data &s. If &; is corrupted, we hope a (>x) can neutralize the influence of &..
Another important machine learning task using bilevel optimization the meta-learning (Franceschi let al. 2018 Lorraine et al. 2020b). Suppose we have N different training tasks. The jth task
* "Use footnote providing further information about author (wehpage, altemaatve address) not for ac- knowledging funding agencies. Funding acknowledgements go at the end of the paper.

---

Published as a conference paper at ICLR 2024
(a) MNIST
(b) FMNIST
(c) CIFARIO
Figure 1: Overfitting in data cleaning tasks
has pair of training and validation data sets {()),: )., The j-th task is to train a network fwy, X; g), where & is an input, wy denotes the parameters for task j and À denotes the parameters shared for all N tasks. The goal of meta-learning is to find good shared parameters A. To this end, the following bilevel model is considered:
min Σ ((ee,,A:jjs s.t. w(1) E
Arg min
Σ A(Wt,):Dy.rh
N
wrt (101 (1))))))
where 1,(10,, A; D) is the loss of the network f for the jth task on a data set D.
As shown in ① and ( bilevel training tasks more complex than single level training tasks. Thus, bilevel optimization models can be sensitive to changes in the input data and parameters. In addition, bilevel training tasks usually suffer from limited data sets. As evidenced by Figure [ in the data cleaning problem, overfitting happens since the increase in the training accuracy leads to a decrease in the testing accuracy of the classifier. Regarding meta-learning, overfitting in the model learned through bilevel optimization can result in a diminished ability to learn new tasks. As shown in our experiments section, in classification problems, the training accuracy is high while the testing accuracy is not. Therefore, overfitting is a common challenge in bilevel training tasks and we need to addressing it.
In single level machine learning optimization problems such as empirical risk minimization, there are many ways to redeem the overfitting problem, including dropout, early stopping, data augmen- tation, etc. Among these methods, dropout is highly effective, simple to apply and computation- ally efficient, (Srivastava et al. 2014; Labach et al. 2019). Proposed by /Srivastava et al. (2014), dropout randomly drops neurons of the neural network during training, leading to only a portion of has been successfully applied to fully connected networks (Ba & Frey 20133] Srivastava et al. (0144). convolutional neural networks (CNNs) (Wu & Gu 2015 Srivastava et al. 2014; Park & Kwak (017)), and recurrent layers in recurrent neural net- works (RNNs) (Pham al. 2014 Zaremba et al. 2014). In this work, we investigate how dropout
parameters being updated in each epoch.
behaves when adopted to solving the bilevel optimization problem.
Since the dropout only randomly zero out part of the neurons in a network, we propose a bilevel optimization model that characterizes this randomness. Since current existing bilevel methods are not directly applicable
this new model, we use representative bilevel method from Li et al. 2022) as an example to see how existing bilevel methods can be adapted to this new model. We then investigate theoretical convergence guarantees of the resulting dropout bilevel method. Existing analysis of the dropout method from optimization perspective is very limited. As far as we search, only (Li al. 20166] Mianjy & Arora 2020 Senen-Cerda Sanders, 2020) Senen-Cerda & Sanders (022) investigated this direction previously. However, these works only focused single level optimization problems. For bilevel problems, there are no theoretical convergence guarantees for dropout methods. In this work, we fill in this gap and study the convergence properties for dropout bilevel method. The challenges in analyzing the dropout bilevel method, which the single- level model considered in (Senen-Cerde & Sanders 2020) does face, include how to analyze the hypergradient. Analyzing the hypergradient is hard to analyze in two ways: 1. it is related to the solution of the lower level problem; 2. both upper and lower level objectives composed with random dropout masks.
Our contributions are summarized follows:
We form a statistical bilevel optimization problem that includes the distribution of the dropout masks. To solve this problem, we propose dropour variant of existing method
2

---

Published as a conference paper at ICLR 2024
from (i et al. 2022) as an example of bilevel method. We investigate the inductive lations between the variables that are used in the dropout bilevel method to approximate the hypergradients of the outer-level objective. These variables are affected by the dropout masks, and the variance term in the inductive relations is affected by the dropout rates.
We show that the complexity of the dropout bilevel method depends on the dropout rate. Unlike the complexity of the bilevel methods for the bilevel model without dropout, the complexity of the dropout bilevel method for the dropout bilevel model has additional er- rors that are only brought by the dropout. The challenge in this analysis is how to handle the distributions of dropout masks that appear when estimating the error of the hypergradient the upper-level objective.
We apply the proposed method to data cleaning problems. In our experiments, we observe the overfitting problem. In addition, observe that the proposed dropout bilevel method redeems the overfitting problem. We also observe that the dropout bilevel method con- verges. Furthermore, we observed that accuracy changes across iterations are more stable when using dropout.
1.1 RELATED WORK
Theoretical properties of dropout methods for single-level optimization problems have been investi-
gated in (Baldi & Sadowski 2013 Gal & Ghahramani 2016 Zhai & Wang 2018 Li et al. 2016).
In particular, Baldi & Sadowski (2013) introduced a general formalism for studying dropout and
use to analyze the averaging and regularization properties of dropout. Gal & Ghahramani (2016)
and (Gal & Ghahramani 2016) formed dropout training approximate Bayesian inference deep
Gaussian processes and studied the model uncertainty. (Zhai & Wang, 2018: Gao & 'Zhou 2016
Wang et al.
2019) studied Rademacher complexity bound of the dropout method.
Li et al. (2016): Mianjy & Arora (2020); SSenen-Cerda & Sanders (2020); SSenn--Cerda & Sanders) (2022) investigated the convergence of the dropout method for single level risk minimization tasks. Li et al. (2016) view dropout as a data-dependent regularizer added to the training loss. In contrast, our approach involves forming a stochastic minimization model that treats dropout masks as a ran- dom linear transformation within the training loss. Mianjy & Arora (2020) considered dropout method for 2-layers network. When the loss function logistic loss (convex) and the activation function is Relu, they provided the convergence rate of testing error. In this work, we consider general multi-layer neural networks and investigate training error. In addition, they assume data distribution is separable by margin in particular Reproducing Kernel Hilbert space, while we do not have assumptions on the data distribution. (Senen-Cerda & Sanders (020) is closely related to work. They studies the convergence properties of dropout stochastic gradient methods for minimizing the empirical risks of multiple fully connected networks. Different from (Li et al.) [016]]
Mianjy & Arora| 2020 (Senen-Cerda & Sanders 2020 Senen-Cerda & Sanders 2022). ), we focus on bilevel training tasks.
Popular methods to solve bilevel optimization problems in machine learning have been proposed in (Franceschi et al., 2017]] 2018 Finn et al., 2017aa Li et al., [002 Gould et al.; 2016 Lorraine ctt al. [2020a] Bae & Grosse, 2020). The major of them are gradient-based methods. They can be Turther divided into two types based on the way they approximate the hypergradient. The first type
Finn et al. 2017a] Liu et al. 2020) Ghadimi & Wang 2018 Ji et al. 2021 Rajeswaran et al. 2019). and the second type is approximate
iterative differentiation (ITD) (Franceschi al. 2017
2018
implicit differentiation (AID) (Chen et al.) 2022 et al. 2021 Li et al., 2022 Gould et al., 2016 Lorraine et al. 2020a). As far as we know, there is no existing work in bilevel optimization that considers the overfitting problem, let alone analyzing the dropout bilevel method for this problem. In this work, we select an ITD method with a relatively simple structure as example to investigate how dropout affects bilevel training tasks.
2 PRELIMINARIES
In this paper. we denote R" the n-dimensional Euclidean space with inner product (:--) and Eu- clidean norm || |]. We denote the spectrum norm of matrix A E Rm as ||A|| and the Frobe- nius norm of A as || For any matrices A and B, denote trace( A B) = (A, B). For
3

---

Published as a conference paper at ICLR 2024
ramdom variable & defined probability space (E,E, P), we denote its expectation Eş. Given an event A and a function f of ç, the conditional expectation of & is denoted as (g) For a function F IR - Thm we denote the function F(x,, y) with respect to y for a fixed as F(z,-) and denote the function F(z,y) with respect to z for fixed y as ().y). For a dif- ferentiable function f. we say x is an stationary point of f if [V || I E. For a twice dif- ferential function F Rm R + R, we denote F=F(x, y) and V F2, y) the partial gra- dients @F(a..) and @F(2.w) correspondingly. We denote F2 y) as the Hessian matrix of F, By VayP y) := 8² F(z.)) and VnFF(x,)) v) 8'F(w.
By
Byjy For multiple valued differentiable func- tion R=" + R" we denote its Jacobian at x as ((())). With a little abuse of notation, given distribution P. let g(2; {) be function that depends on E R" and data point & ~ P.
Ozoy
In general, bilevel optimization training task formed as follows:
min F(), (A)) = FA, w (A); Deal), s.t. ((A) arg min G(A, w) == G(), w; Dir), wE
(3)
AER
where Drat is a validation data set, Der is a training data set, F: R + R and G RnAto R differentiable functions
As mentioned in Section 2 there are various approaches to solve (3). Due to the nested structure of bilevel problems, the methods for (3) are inherently complex. To better understand how dropout
implemented methods for ( we use the fully single loop algorithm (FSLA) proposed in (Li example investigate dropout. This method has relatively simple formulas at each iteration and low computational cost per iteration. Step in FSLA is one step of SGD for the lower level problem and Step 9 generates an approximation of the hypergradient of the objective in the upper level.
et al. 2022)
2.1 DROPOUT
Let fw:; be an l-layer network, where w the weight and & an input data. In particular, we assume
(4) where Wi E RR x Ro: is the parameters in the ith laye a; is the activation function in the ith layer. We let weight matrices, where (wi)s is the transpose of the jith row of w, Thus w is an no-dimensional vector, where = Σ ni. Σ oz Let / be loss function. When training f with loss function the forward pass is
[w:;)) == [(w;;) = ar (wr( (ar (wy ................................ (wi ())))
((w1)1,... (wi)m,..... (w))1....
(wy) mz) be the collection of all rows in all
F(w; {) = (w::():):
(5)
In the ith layer, the forward pass of an input E, E 18 using dropout can be formed as
r = m, aw(CC))
(6)
where En &, represent the Hadamard product of two vectors, and m, E IR's with (m²) Bernoulli(p:) with p. E [0, 1]. my is called dropout mask at the ith layer. Note that this formation of dropout masks is more general than the original dropout masks proposed in Srivastava et al. 2014). In Srivastava et al. (2014), the dropout rate is the same for all rows of weights in the same layer. Here, we allow the drop rate of each row of the weight matrix to be different.
Note that the function values of many well known activation functions at 0 is O. For example, tanh, centered sigmoid and Relu has this property. Through out this work, we make the following assumption.
Assumption 1. For any a; used in (0) 0.
Under assumption r; in (6) can be alternatively formed as
(m²)1
r; = am,, (w.Ea) = a;
w
(m²) m..
'ith a little abuse of notation, without misunderstanding, we denote the F(A, w(A): Deat) as a function F(X) and denote G(A, w; Der) as a function G(X, w²). Forr simplicity, we include the parameters of bias in ue with corresponding fixed input of 1.
4

---

Published as a conference paper at ICLR 2024
(m:)1
0
0
Denote diag((m:))...... (m;)).))))....))
Recalling that for any layer i,
(m n.
(m:), Bernoulli(p:). The right hand formula in means the dropout method randomly pick the jth row of the weight matrix w, with probability ps and sets the rest to Thus, given a loss function I, under Assumption [
the forward pass with dropout in fact calculates
[mfm, E) = F(mm;;)),
where I' defined in 5 and
(8) As we can see, the jth diagonal element mij Bernoulli(p:) when j 1Ex ne + I,..., Ex ]. Note that in above discussion, each row of the weight matrix in each layer is always viewed as element. For notation simplicity, without loss of generality, in the rest of this paper, when we mention the weight of a network, we view the weight matrix the ith layer as a vector of IR where n; is the number of outputs in the ith layer. Then, we view w, the collection of all rows of all layers, as vector in I
= diag ((m))......m
(mi)......
(mi)ne)
3 DROPOUT IN BILEVEL PROBLEMS
As introduced in Section 2.1] the objectives are the composition of them with the dropout masks. Thus, propose the following variant of (3) that considers the distribution of the dropout masks:
min F(A)= =- Em,, m.. F (mxx, m.. (A)), AER
(9)
s.t. w(A) E arg min G(A, w) Em G(m ww wER"
where F' and G' are the same as in (3, m. and m. random diagonal matrices with (mx)a sernoull((pi.)) (m ~ Bernoulli(pi.u respectively. This idea of modeling with the dropout masks is also considered in analyzing dropout SGD for the single level training tasks in (Senen- Cerda & Sanders) 2020). For bilevel learning tasks, this model is new. We add dropout masks on both upper and lower level objective functions include the application where À and
both
weights of a network, Franceschi et al. (2018): Zügner & Günnemann (2019); Finn et al. (017a);
Snell et al. (2017).
Now we adopt the dropout bilevel method. based on FSLA. Before presenting a general form of FSLA with dropout, let's consider applying it the example mentioned in (I). The data clean- ing problem
[ is case of (3) with F(), w()); Dratt [Naat Σier (([(w(()): my)s ms) and
N. Eer (.::)) E). As we illustrate in Section 2.1 suppose m is a dropout masks added in the forward pass of the network used in (T) in each iteration. Then the forward pass of the upper level objective used in FSLA becomes F(X, mw((): Deat), where m is mw;; D.r).
G(A,, wi D.r)
defined in (8). The forward pass of the lower level objective used in FSLA becomes G(),mw; By the chain rule, the backward pass needed FSLA calculate the follows:
V VF((), (F((),,w(() D Doat) mVV(()) mw)():;( (A); VG(, m()); D.. (A, w) = m GV mww; Dir); V²G(, m()): D
(10)
Dtr (A, w) = mV²G mw; D.r)m.
Based this, we present Algorithm We add projection in step 8 to avoid blow up. This is also important in the theoretical analysis. In the next section, we analyze the convergence of Algorithm
ANALYSIS OF ALGORITHM
A challenge of analyzing p which the single level model considered in (Senen-Cerda & Sanders 2020) does not have, is how analyze the hypergradient (A(). F(()) is hard analyze due to
5

---

Published as a conference paper at ICLR 2024
Algorithm FSLA with dropout
Input: 0, 0, 20, w., dropout rates (pr.))-1 (Pr.) VF(A., w 1 1)).
2: for k = ...... T. do
3: ag Vi 7 = 704; 8K Bou:: Ue = mark-
4: Generate random diagonal matrices T T. my and m with (TT (m
Bernoulli(p..) (ma)as (m=))ss Bernoull((ps..)
5: Sample & Der, T7th Deal.
6: wh+1 wt pmm V Gm my
7:
gh+1 3m V Pm m w ":n")) (I Bam A V G(m m )m wh. m wt 2 ym uk+l1
8:
otl Pro
9:
T jo m af (m m
mw w 1:111) 111%)~myy Mn X Gl A mn
10:
d = (A( wht1 eltt) + (1 - (d -F(A-11 , r())).
11: Ak+1 = A dht1
12: end for
its relation with the solution of the lower level problem and the fact that both upper and lower level objectives are composed with random dropout masks. To analyze this and Algorithm for (9) we first make the following assumptions that are standard in bilevel optimization literature.
Assumption 2. Let F(X, w) be defined in (3. Suppose the following assumptions hold:
(i) Suppose F Lipschitz continuous with modulus Lp.
(ii) For any fixed X, VxF(A,)) and F(w,,. are Lipschitz continuous with modulus L and T.g
(iii) There exists CF O such that o. IIF
IV w)) S C for any X and
(iv) For any fixed w. Fw w) Lipschitz continuous with modulus L 0.
Assumption 3. Let G(X, w) be defined as in (3). Suppose the following assumptions hold:
(1) G twice continuously difserentiable. VG w) is Lipschitz continuous with modulus La-
(ii For any G(X,,) ) is strongly convex with modulus μ.
(ili) For any G2)G(),) )) is Lipschitz continuous with modulus LGv.
(iv) For any w, Vi, w) is Lipschitz continuous with modulus Lea.
(v) There exists CG such that max,, V7, 1I S for any X and w.
Denote M ma
and FM Fm mm u0 (A)) Then
0
Mb
VF(A) Em,.m. VFM(A).
(11)
Under Assumptions and
and it is easy to see that using the chain rule.
FA) mA VA F(mx), m))( + (ww(A)) m V F(mx,, mA)), where (W)()) the Jacobian of wA)). The next proposition gives a closed form of J(((A)) and its property.
(12)
Proposition 1. Consider and suppose Assumption 3 holds. Denote W
and M
w
mx
Let p.,. min; ps Then & is strongly convex with modulus pn, and the following
my
equalities hold:
Vā w) EMVG((M)) VOG w) EMV*G M:

---

Published as a conference paper at ICLR 2024
J(w(A)) = -v G(A, Vin C(A, w()))
(13)
In addition, for any X, it holds that w(X) is Lipschitz continuous with some modulus L Also, for any it holds that [[ s Cc.
Based on Proposition] we can estimate the variance of VV w).
Lemma 1. Let G be defined as in (3). Let mn and mhe are diagonal matrices with (mx)is ~ Bernoulli(py (ps.)) and (mw)is Bernoulli(wwa) Let pı Pi,A if fI,..., nX} and ps Pi ifi {ml + +.....,,y ny + n- Then
VIG (W) VNG (W) S Σ P.Ps (1 -PpP)) Cr.
(14)
Next, we show how the update of the lower level parameters behaves. Since Algorithm stochastic gradient and Hessian. We add the following assumptions.
I uses the
Assumption 4. Consider 9 Let g be randomly picked from Dir and nj be randomly picked from Dual. Suppose E,, V P( n)
VxF(A, w) and Eg (A,, w; G. w). Suppose
E, F w; m) F w) 12 S 0². Ee V G w;; E) G.G o², and for any
J. E V G(), () V G(A, w))l & oh
Now we show the inductive relations of {w²} generated by Algorithm
Lemma 2. Consider (9) Suppose Assumptions 2 and 4 hold. Let {(>*, w be generated by
Algorithm| ! Let 7/ in Algorithm The small enough such that yp < 1. Let Pw == max; Pi,w- Then there exists (0, 1) such that
E| W
un( u=(>*) (222) S [E|| w(( 0aa
O
(15)
Remark 1. Note that w(X* is the true solution of the lower level problem given X². This induction has additional errors related to the updates of the upper level variable X and the variance. Note that the variance term in (5) decreases linearly with the maximum dropout rate.
Next, present the inductive relations of {u*} generated by Algorithm with 12 it is easy to see that
In fact, combining 13
V FM)(A) mA V F(mA, m()) - V2 G(), w w()))mm....
(16)
where Uman.. =
2 a00 ((),w(1)) my V. wl)) This implies that
Um Bm V (m,A, m m ((A) + (1 G(, (()))())) Um..d.
Comparing the above equation with the update of xt1 in Algorithm I we see that w is an approximation um The next theorem shows how the difference u and Umyy. varies with iteration & and the dropout rate.
Theorem 1. Consider (9. Suppose Assumptions
2 and 4 hold. Let {(A", w²) be generated Denote Vk. w V ---11 Denote Px = max, Pt.x. Px == min; PtA
by Algorithm !
P.. = max; Piw and p... == min; Pl.w- Suppose r 2 C. := 1.2 Then there exists (0, 1) such that
EIl S (( OZP,. + Oa ((P 3 O(pxPm0 )8)
- - w(
Remark 2. The last term the above theorem are introduced by the dropout of the network and disappear when dropout is applied to the network parametrized by
Now, based on Theorem D and Lemma 2] we analyze the convergence of Algorithm [ Thanks to
IT]. to find an stationary point to (9) such that [A)]] < c, it suffices to find that satisfies
(A)

---

Published as a conference paper at ICLR 2024
(a) MNIST
(b) FMNIST
(c) CIFARIO
(d) MNIST
(e) FMNIST
CIFARIO
Figure 2:
Results from data cleaning. The first line reports how training and testing accuracy change as the number iterations increases. The second line details the training and testing accuracy the final iteration when different dropout rates are applied.
hold. Let {ww", ,²1) be generated by Theorem 2. Consider 99. Suppose Assumptions 2 and 4
Algorithm 7
Suppose is bounded below by F. Then there exist small ö in Algorithm [
such that
T-1
In( +1)
1n(T+1)
p PxPw
T F
To +
Lu
(o2
P.
VT
In(T + 1) VT
(2 - prm) + Pwo (1 pm)))
Pm
Remark 3. The first term shows that the convergence rale is slower when P2P is small, which can be conformed in our experiments Figure The last term in the above results introduced by dropout, and it disappears when P, = = I, i.e., no dropout is applied.
On the other hand, have to point that the convergence of EV F (X*) does not imply that of F(A). Consider a simple 2-dimensional case. Let f(11, 222 = 109=2 + 1000 Let x² = -(xt, x²) be with at = 10000"+ +200001 -P)p and x² = k. Let m = diag(mi,ms with my = Bernoulli(p) and ~ Bernoulli(p). Then limp-+o0 Em ff(m==)) = O. How- (-1000p² 2000(1 - plp 1000, 0). Therefore, the convergence of VEm (max) does not necessarily imply the convergence of fz=*), the error between V V((x*) and E (mnx*) can not be closed unless approackes
&
ever, lima-opo vf(aa)
Therefore, there are additional errors brought only by the dropout. This example implies that the dropout method may not be optimizing the original bilevel problem. This is expected because when the original training loss is optimized, the model fits the training data too well and this will increase the chance of overfitting.
EXPERIMENTS
In this section, Algorithm on data cleaning tasks TT) and meta learning problem 2 The experiments were conducted on machine equipped with an Intel Xeon E5-2683 CPU and 4 Nvidia Tesla P40 GPUs.
Data cleaning We followed the approach outlined in (Srivastava et al. 2014) and utilized fully connected network. The experiments were performed using the MNIST dataset (LeCun al., 2010). The network architecture employed was 784- 144-100-1222-248--1111
with ReLU activation functions used for all hidden units. To introduce unclean data, we randomly mislabel 60% of the training data. When training on MNIST and FMNIST, we use fully connected network with the architecture 784 - 1024 1024 2048 10 and employ ReLU activation functions for all hidden units. In all experiments conducted on MNIST and FMNIST, we set the dropout rates of all layers to the same value, denoted as Dropout is only applied when updating the lower level parameters. We set y 0.01 and train 10000/10000 iterations for 5000/ 100000000000000000000000000000 MNIST/FMNIST data points. When
8

---

Published as a conference paper at ICLR 2024
(a) Layer with p = 0.8
(b) Layer 2 with p = 0.5 (c) Layer with p = 0.5
(d) Layer
(c) Layer 2
(f) Layer 3
Figure 3: Results from few-shot learning on the Omniglot dataset. The first line reports how train- ing and testing accuracy change as the number of iterations increases. The second line details the training and testing accuracy in the final iteration when different dropout rates are applied to only one layer of the network.
training on CIFARIO, we use 4-layer convolutional neural networks + fully connected layer. In all CIFARIO experiments, we apply dropout after each convolutional layer, using consistent dropout rate of p. We set 7 = 0.1. We train 40000 iterations for 40000 data points.
Meta-learning We conduct experiments with the few-shot learning task, following the experimental protocols of (Vinyals et al., 2016), we performed learning tasks over the Omniglot dataset. We set train/validation/eest with 102/172/423, respectively. We perform 5-way-1-shot classification. More specifically, we perform 5 training tasks (N 5). For each task, we randomly sample 5 characters from the alphabet over that client and for each character, and select 1 data points for training and 15 samples for validation. We use a 4-layer convolutional neural network Each convolutional layer has 64 filters of 3x3 and is followed by nochaioomallizttoon layers (Finn) et al. 2017b). The parameters of convolutional layers are shared between different tasks (A) and the last linear layer the task specifci parameter Wy. In each experiment, we only add dropout one CNN layer. In all experiments, we let B, and y in FSLA and Algorithm I be 0.05, 0.1 and 0.8 respectively.
fully connected layer.
Results We report the results in Figures[aad]3 In the first line of both figures, we plot the accuracy against the iteration in each training progress. In the second line of both figures, we plot how different dropout rates affect the training and testing accuracy. Figure 2 shows when the training accuracy increases, the testing accuracy decreases. This observation indicates the occurrence of overfitting in data cleaning tasks. The proper addition of dropout during training can enhance testing accuracy in response. On the other hand, Figure 2 demonstrates that training the network with dropout leads to convergence accuracy of the method when adding dropout with different rates on the different layers. As we can see, adding proper dropout to any layer improves the testing accuracy.
higher testing accuracy and greater stability. Figure 3 shows the
6 CONCLUSION
In this paper, we explore the application of dropout bilevel training tasks. We propose stochastic bilevel model that is dependent on the distribution of dropout masks. We adapt an existing bilevel method with dropout. We analyze the convergence properties of the resulting method for the pro- posed model. We investigate the inductive relations of the variables attributes to an approximation of the hypergradients. In addition, we show how the dropout rates affect the complexity. We be- lieve that other ofate-of-art bilevel methods can also be adapted to address the stochastic bilevel model with random dropout masks, and our convergence analysis serve as the first example for such adaptations.

---

Published as a conference paper at ICLR 2024
REFERENCES
Jimmy Ba and Brendan Frey. Adaptive dropout for training deep neural networks. In C.J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K.Q. Weinberger (eds.), Ad- vances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013. URL Ittpp:://proceeellggg. cc/pppperrccppppr.eepcppe fttpppppprr..llespppe///1/11 Ei File/ htse://222aaadt1113300ddaaabbbbs
Juhan Bae and Roger B. Grosse. Delta-stn: Efficient bilevel optimization for neural networks using structured response jacobians. In Hugo Larochelle, Mare' Aurelio Ranzato, Raia Hadsell, Maria- Florina Balcan, and Hssuan-Tien Lin (eds.), Advances in Neural Information Processing Systems Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, De- cember 6- 12, 2020, virtual, 2020. URL http:tttttt///rrceeeddggn neweeeerddnnnnneerppp.. Cc cc/paper/ 22020/hash/ (50110/1855444777777700005533333 AAstraa htm
33:
Pierre Baldi and Peter J. Sadowski. Understanding dropout. In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger (eds.), Advances in Neural Information
Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pp- 2013. URL. /tr
htc.pps.ccccppee wttpppppp///..//eeeeepeeeep wwd.bf 0aw.://00..0..000000aaaaaaaa00.0 Abstract html
Ziyi Chen, Bhavya Kailkhura, and Yi Zhou. A fast and convergent proximal algorithm for regular- ized nonconvex and nonsmooth bi-level optimization. 2022. URL
18550/arxi10 2203. 16615
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August, 2017a
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August, 2017b.
Luca Franceschi, Michele Donini, Paolo Frasconi, and Massimiliano Pontil. Forward and reverse gradient-based hyperparameter optimization. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August, 2017.
Luca Franceschi, Paolo Frasconi, Saverio Salzo, Riccardo Grazzi, and Massimiliano Pontil. Bilevel programming for hyperparameter optimization and meta-learning. Jennifer G. Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pp. 1563-1572. PMLR, 2018 URL www.hhtpp/// pestp://proceedinn mlr. Batess//88//raaccccchhe
htmi
Yarin Gal and Zoubin Ghahramani. A theoretically grounded application of dropout in recurrent neu- ral networks. In Daniel Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Ro- man Garnett (eds.). Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spuin, pp.
1019-1027, 2016. URL ht http:tttpppppceeeeeeeeeee 0TT500977000011100ccal1112222223 www.sabb-bbbtttttt......aa... html
neurips. wttpcc/pp/pee//ppeeeeepppppp
Wei Gao and Hui-Hua Zhou. Dropout rademacher complexity of deep neural networks. Sci. China Inf. Sci., 590))00721044 2016. doi: 10.10007/114332-0155555
URL https:
Saeed Ghadimi and Mengdi Wang. Approximation methods for bilevel programming. 2018. URL
https::/arxian http://as///......0
Stephen Gould, Basura Fernando, Anoop Cherian, Peter Anderson, Rodrigo Santa Cruz, and Edison Guo. On differentiating parameterized argmin and argmax problems with application to bi-level optimization. 2016. URL. http::ttt:::aatt
w0w.aa/////////.// 05447
10

---

Published as a conference paper at ICLR 2024
Kaiyi Ji, Junjie Yang, and Yingbin Liang. Bilevel optimization: Convergence analysis and enhanced design. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July, 2021.
Alex Labach, Hojjat Salehinejad, and Shahrokh Valaee. Survey of dropout methods for deep neural networks. CoRR, abs/1904.13311 2019, /ttp:/:::/arx abs/ abs/1904............... 13310
Y. LeCun, Cortes, and C. Burges. Mnist handwritten digit database. 2010. URL http:// yann. lecun. com/
exdb/ com/exdb. 2.
Junyi Li, Bin Gu, and Heng Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty- Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, Febru- ary 22- March 1. 2022.
Zhe Li, Boqing Gong, and Tianbao Yang. Improved dropout for shallow and deep learning. In Proceedings of the 30th International Conference Neural Information Processing Sys- tems, NIPS' 16, pp. 2531-2539, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
Risheng Liu, Pan Mu, Xiaoming Yuan, Shangzhi Zeng, and Jin Zhang. generic first-order algo- rithmic framework for bi-level programming beyond lower-level singleton. In Proceedings the 37th International Conference on Machine Learning, ICML 2020, 13-18 July, 2020.
Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit diffcrentiation. In The 23rd International Conference on Artificial Intelligence and Statis- fics, AISTATS 2020, 26-28 August, Online [Palermo, Sicily, Italy]. 2020a.
Jonathan Lorraine, Paul Vicol, and David Duvenaud. Optimizing millions of hyperparameters by implicit differentiation. In Silvia Chiappa and Roberto Calandra (eds.), The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]. volume 108 of Proceedings of Machine Learning Research, pp. 1540- 1552. PMLR, 2020b. URL http:///////rroeedigggs... mlr wtw.a/ppreees/ooooorraaaaaaa html
Poorya Mianjy and Raman Arora. On convergence and generalization of dropout training. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neu- ral Information Processing Systems, volume 33, pp. 21151-21161. Curran Associates, Inc.,
2020. URL https: File/ htle://ee50000033322222666666222
files/paper/2020.
pdf
Sungheon Park and Nojun Kwak. Analysis on the dropout effect in convolutional neural networks. In Shang-Homm Hong Lai, Vincent Lepetit, Ko Nishino, and Yoichi Sato (eds.), Computer Vision ACCV 2016, pp. 189-204, Cham, 2017. Springer International Publishing. ISBN 978-3-319-54184-6
Vu Pham, Théodore Bluche, Christopher Kermorvant, and Jérôme Louradour. Dropout improves recurrent neural networks for handwriting recognition. In 2014 14th International Conference on Frontiers Handwriting Recognition, pp. 285-290 2014. doi: 10. (099/CHHH.2014.55.
Aravind Rajeswaran, Chelsea Finn, Sham M. Kakade, and Sergey Levine Meta-learning with im- plicit gradients. In Advances Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurlPS 2019, December 8-14, Vancouver, BC, Canada, 2019.
Albert Senen-Cerda and Jaron Sanders. Almost sure convergence of dropout algorithms for neural networks. CoRR, abs/2002.002477, 2020. URLhhttpss///axxir.or///////////
Albert Senen- Cerda and Jaron Sanders. Asymptotic convergence rate of dropout on shallow lin- neural networks. SIGMETRICS Perform. Eval. Rev, 50(1):105-106, jul 2022. ISSN 0163- 5999. doi: 10.1145/55733333330666.
URL htttp:://dbi.ozgg/// 11t.://0000000011155515153353330
8530965
11

---

Published as a conference paper at ICLR 2024
Amirreza Shaban, Ching-An An Cheng, Nathan Hatch, and Byron Boots. Truncated back- propagation for bilevel optimization. The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April. Naha, Okinawa, Japan, 2019.
Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Informa- tion Processing Systems 2017, December 4-9, Long Beach, CA, USA, 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way prevent neural networks from overfitting. J. Mach, Learn. Res., 15(1): 1929-1958, 2014.
Oriol Vinyals, Charles Blundell, Tim Lillicrap, Koray Kavukcuoglu, and Daan Wierstra. Match- ing networks for shot learning. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, Barcelona, Spain, 2016.
Haotian Wang. Wenjing Yang, Zhenyu Zhao, Tingjin Luo, Ji Wang, and Yuhua Tang. Rademacher dropout: An adaptive dropout for deep neural network via optimizing generalization gap. Neu- rocomputing, 357:177-187, 2019. doi: nww.://..ccmmmm19....0 101/600.0016/ neucom. 2013 9 .5.008
URL https:://oii.
Haibing Wu and Xiaodong Gu. Towards dropout training for convolutional neural networks. Neural Networks, 71:1-10, 2015. ISSN 0893-6080. doi: http::/tbi.org//.0011j...eutt
2015.07.007. URL https:/
3089360801500144
Zhuoran Yang, Yongxin Chen, Mingyi Hong, and Zhaoran Wang. Provably global conver- genee of actor-critic: A case for linear quadratic regulator with ergodic cost. Ad- vances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL
ottprpcceeeeeeeeeeeeerrso...pppp File 711499933aa22bb02222236666662663
Wojeiech Zaremba, Ilya Sutskever, and Oriol Vinyals. Recurrent neural network regularization. CoRR, abs/1409/ 400.23399, 2014. URL/pttp.//arxxi..cc/////////
Ke Zhai and Huan Wang. Adaptive dropout with rademacher complexity regularizaonnlliiiiiiiinoo In Interna- tional Conference Learning Representations, 2018. URL https:://openeeeieeww.
Eorum? 1==ummmd==u
Daniel Zügner and Stephan Günnemann. Adversarial attacks on graph neural networks via meta learning. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019.
12


Reviews:
[{"review_id": "oCgE2TpDUR", "reviewer": "Reviewer_Kt41", "summary": "This paper proposes a dropout method to address the issue of overfitting in bilevel training tasks. The authors provide theoretical convergence guarantees from an optimization perspective and demonstrate its effectiveness through experiments, using data cleaning as an illustrative example.", "questions": "- In addition to the theoretical analysis, literature [1] also investigates the impact of dropout rate on bilevel optimization. However, what are the other distinguishing factors between these studies?\n- What is the relationship between the objective function after incorporating dropout and the original objective function? \n\n[1] Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians. Juhan Bae, Roger Grosse. NeurIPS 2020.\n\n[2] Bilevel Optimization: Convergence Analysis and Enhanced Design. Kaiyi Ji, Junjie Yang, Yingbin Liang. ICML 2021.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is generally well-written and easy to follow. It investigates a relatively unexplored area and aims to analyze the dropout bilevel method for addressing this problem. Another advantage is the detailed theoretical analysis provided in the paper.", "weaknesses": "- Based on the results figures, it appears that early stopping can effectively resolve the issue of overfitting, even though the accuracy without dropout is higher.\n- The theoretical analysis in the current context is not particularly challenging. In contrast, in many stochastic bilevel optimization studies, such as [2], a more comprehensive framework is presented, along with convergence rate guarantees for stochastic bilevel optimization. You can try to answer the extra challenge in these theoretical literatures [2].\n- The experimentation conducted on data cleaning is insufficient, and it would be better to observe more results on other bilevel optimization tasks.\n- Each figures should be accompanied by a brief caption.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "p6oCa4K9iR", "reviewer": "Reviewer_hA9j", "summary": "This paper brings dropout to bilevel optimization to avoid overfitting issues. The author forms a statistical bilevel optimization problem that includes the distribution of the dropout masks and propose a dropout method to solve it. To analyze the convergence of the proposed method, they delicately quantify the bias induced by the dropout. The empirical performance of the proposed method is tested on the data hyper cleaning task and shows that dropout efficiently improve the test accuracy and is more stable.", "questions": "See weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper incorporates the dropout technique in the bilevel optimization to avoid overfitting. The idea is new to the bilevel optimization community and has been tested effective. \n2. The analysis of dropout by modeling the dropout mask as a stochastic distribution is thorough and novel. \n\nOverall, I felt this is a solid and novel paper for bilevel optimization.", "weaknesses": "1. The motivation of considering the dropout in bilevel optimization could be explained more. For example, why avoiding overfitting is important in bilevel optimization, that is, does bilevel structure exacerbate the overfitting issues? \n2. The convergence analysis is built up on the stationarity measure $\\frac{1}{T-1} \\sum_{k=1}^{T-1} \\mathbb{E}\\left\\\\|\\nabla F_{M^{k}}\\left(\\lambda^{k}\\right)\\right\\\\|^{2}$, which is the dropout bilevel objective. To fortify the analysis, it is conceivable to draw connections between this measure and the one based on the original bilevel objective $\\frac{1}{T-1} \\sum_{k=1}^{T-1} \\mathbb{E}\\left\\\\|\\nabla F\\left(\\lambda^{k}, w(\\lambda^{k})\\right)\\right\\\\|^{2}$. This could elucidate the relaxation error introduced by dropout and establish a guarantee for the original objective's convergence, thereby validating efficiency of the dropout approach more rigorously.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "S3j0qOxj5t", "reviewer": "Reviewer_fs9w", "summary": "The paper uses dropout methods for bilevel training tasks. Bilevel optimization problems consist of two intertwined optimization problems and can be particularly sensitive to small changes, especially when data is limited. The study introduces a bilevel optimization model that considers the distribution of dropout masks and examines how varying dropout rates impact the hypergradient of this model. The authors adapt an existing bilevel method to incorporate dropout and provide theoretical convergence guarantees for this new approach. Empirical tests on data cleaning problems show that this method can mitigate overfitting.", "questions": "Beyond data cleaning, how might this method be applied to other machine learning tasks, especially those that inherently involve bilevel optimization?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Incorporating dropout in bilevel training tasks is a novel approach, offering a new method to combat overfitting in such tasks. The authors study the convergence properties of the introduced method. The study offers empirical proof, especially in the context of data cleaning problems, demonstrating the efficacy of the proposed method in reducing overfitting. The paper paves the way for adapting other state-of-the-art bilevel methods to account for dropout, making it a foundational study for further research in this direction.", "weaknesses": "The use of datasets like MNIST, which is relatively small and simplistic, might not fully showcase the potential or limitations of the proposed method in real-world, complex scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8SjJOErGTH", "reviewer": "Reviewer_mkUf", "summary": "This paper points out that overfitting is a common challenge in bilevel training task. To address this issue, the author proposes the dropout mask for the bilevel optimization problem. Specifically, the author proves the convergence of dropout bilevel methods theoretically and empirically shows that proposed method mitigates the overfitting issue.", "questions": "Can you discuss the technical difficulties of applying dropout analysis into bilevel optimization problem? That will be helpful to understand the theoretical contribution.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "Overfitting issue has not been fully investigated in the bilevel area and this paper proposes the new method to address it.", "weaknesses": "1. It looks like this paper studies data cleaning problem rather than the bilevel problem. Both the introduction and experiments are based on data cleaning context. The authors are suggested to discuss and conduct experiments on other bilevel optimization settings to demonstrate applicability of proposed algorithm.\n\n2. Overfitting issue can be easily addressed by early stopping. The authors are encouraged to make an experimental comparison between proposed dropout and early stopping to demonstrate the necessity of adopting dropout.\n\n3. In Theorem 2, the authors demonstrate that dropout rate \"influences\" the upper bound. However, it only influences the constant term and does not influence the convergence rate. Furthermore, the theoretical results only demonstrates the dropout rates \"influence\" the convergence but does not demonstrates it \"improves\" the convergence rate, even for the constant term. I expect to see inspiration about dropout rate selection from this theorem. The same issue also occurs for the Lemma 1 which is about variance term.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (spotlight)