Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a method for Bayesian Optimal Experiment Design (BOED) which uses a policy network to decide which experiment to perform next based on the history of past experiments and their outcomes. The goal is to enable experiments to be chosen quickly for real-time applications. The method trains a policy network in simulation, and does not require assumptions necessary for a prior method (DAD), which also trains a policy network in simulation but requires additional assumptions such as the availability of a likelihood model of the history given the parameters and policy. The present approach removes this requirement by using variational approximations together with a reparameterization which enables optimization with gradient descent. Experiments are done in a 2D location finding environment, a pharmacokinetics model and an epidemiology simulation. Results show that the proposed method almost matches the performance of an exact method which has access to exact likelihood on the first task, and does better than baselines on all the tasks while being fast to evaluate. 
main_review: This paper is very much outside my research area and I’m not familiar with the current state of the art in the area so my review is an educated guess. From what I can tell the paper addresses an important question, which is being able to perform experiments in real time (they give the example of tracking the source of a chemical spill which I found to be a good motivating example) for a general class of problems which only require the ability to simulate from the environment and compute gradients of the measurement wrt design. This is still not totally general since not all processes are differentiable, which should be mentioned somewhere The paper is very well written and did a good job of making the main arguments accessible. 
limitations_and_societal_impact: Yes, they mention the substantial computational cost of pretraining.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes iDAD, an amortized learning framework for implicit (likelihood-free) experimental design. The iDAD framework extends the previously proposed deep adaptive design (DAD) method (Foster et al., 2013), which is an amortized learning framework for experiment design in likelihood-based (i.e. non-implicit) models, which learns a policy network offline that can then be deployed online for fast experimental design. Additionally, the proposed iDAD framework aims to alleviate the constraint in DAD that requires conditionally independent experiments.
main_review: This paper is written quite clearly and does a good job in providing context for the proposed iDAD method. As such, the contributions of this paper are easy to follow. However, there are a few places where I think this paper could have provided additional details or empirical study, which I summarize below.

I would appreciate more discussion and experimental evaluation on the cost/difficulty of training the policy network to an accurate level under a greater variety of settings. For example, are there conditions, in terms of dimensionality of the input space, dimensionality of the parameter space, query budget or other factors, in which the cost of training an accurate network becomes prohibitive? It would be ideal to include an ablation study that investigates the settings where training becomes difficult, and show the rate at which performance suffers in these conditions, relative to non-amortized methods.

Related to this, I think this paper would benefit from including a discussion (or even some experimental evaluation) on situations where there are (slight) errors with the design policy network—i.e. showing how much will minor errors affect the performance of this amortized method in practice, and detailing any strategies to mitigate any issues caused by these errors.

The iDAD method seems to still have the restriction that one is able to compute the derivative of y with respect to the design point (e.g. via automatic differentiation). While there were a few examples shown of likelihood-free BOED situations where this is possible, it would be great to try and better characterize the scope or types of models that this applies to, and any examples of cases where the framework is limited due to this requirement.

Finally, as the proposed work is motivated by the existing DAD method, I feel that this paper would be improved if there were more specific examples in the introduction of implicit BOED settings where the proposed methods provide advantages over the DAD (and other non-implicit BOED) methods.

—————————————————

**Update:** I thank the authors for their detailed response to my review, and for providing the additional empirical results. I believe these have sufficiently answered all of my questions and concerns that I described above.
limitations_and_societal_impact: This paper includes a section on limitations in the Discussion section, which is a nice aspect of the paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper tackles the problem of adaptive Bayesian experimental design. Specifically, the authors try to relax the conditionally independent experiments and the explicit likelihood of the policy-based adaptive design model proposed by Foster. Code and supplemental materials are provided. 

There are 3 contributions:
- By leveraging the advance in variational mutual information, the authors can establish a suitable likelihood-free objective in the implicit setting. 
- The authors theoretically show that they can obtain a unified information objective without conditionally independent experiments. 
- Based on those insights, the authors propose the iDAD algorithm, which is much based on the previous DAD work. 

main_review: Main Strengths:
- The paper makes a very good connection between the difficulties and limitations of the previous  DAD models and how they derive the idea of new modifications. This shows a very clear motivation of the authors in this paper.
- The idea of using a critic to derive a lower bound for the objective function and optimize this lower bound at the same time with the objective function is interesting, somewhat similar to the development of actor-critic from Reinforce. This mechanism helps to extend iDAD to various applications.
- The authors provide a solid theoretical foundation for their proposed method.  The author successfully eliminates conditional independence which is a restriction of the previous DAD model.
- All the proofs are clear and easy to understand although the idea of the proofs is not new. Overall the paper is well presented and easy to follow; the structure of the paper is similar to DAD (which is not a bad thing). 

Weakness: 
- The method requires a lot of pre-training which can be computationally expensive.
- The performance of iDAD is still worse than that of DAD in many experiments shown. Probably the authors need to reserve more space to do more ablation and analyze why it is the case. For example, when claiming there is a “cost-performance trade-off”, we also want to see when the performances peak, do they beat that of DAD and why?   

limitations_and_societal_impact: The authors did not provide limitations and societal impact directly derived from this work.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: In this work, the authors extend the DAD method by using implicit models to overcome some weaknesses of DAD. The authors further derive several bounds to design re-parametrization gradients. The authors show that the proposed method outperforms existing methods in several Bayesian experimental design tasks. 
main_review: Originality: It is new to use implicit models in the setting of Bayesian experimental design. This work is a novel combination of implicit models and DAD.

Quality: It seems that the proposed method is technically sound and promising. I do not check the correctness of the proofs in the appendix. The experimental results also support the claim.

Clarity: This submission is easy to read and well-organized.

Significance: I think this work extends the scope of DAD and it could be useful for Bayesian experimental design tasks.
limitations_and_societal_impact: yes
review_title: null

Meta-review: The paper is on Bayesian experimental design, building on the recently proposed Deep Adaptive Design (DAD). The paper's contribution is an extension iDAD (I for implicit) that does not require explicit likelihoods, does not require that experiments are conditionally independent, and is fast enough for real-time applications. The paper is interesting and thorough.
Decision: Accept (Poster)