Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes robust model adaptation, which first do a pretraining with adversarial weight perturbation. Then the final result is obtained by iteratively doing model adaptation on the learned model, with a local smoothness penalty objective. The proposed results shows improvements over existing methods.
main_review: The paper is generally easy to follow, the motivation for the added smoothness penalty is clear. The pretraining step is very similar to what is proposed in [46], it would better if the authors could clearly illustrate the differences, if any. One concern is regarding hyper-parameter selection, as the authors have mentioned and also demonstrated in section 4.2. Although the method is relatively stable on several different hyper-parameter settings, the variance on some of the datasets seems large (i.e. Superconductor, HopperController). It seems that the selected alpha and eps values are the best performing ones. It may be good to include the results of other hyper-parameter settings in the main experiments so that it would be a more clear comparison, since the performance of the proposed method and other methods are close in most of the scenarios.
limitations_and_societal_impact: The authors have described the limitations and impact in the paper.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: RoMA is a framework that leverages the local smoothness prior to robustly approximate black-box objective functions. RoMA consists of two steps: 1, train the proxy models (neural networks) to approximate black-box objective functions by perturbing original inputs with Gaussian inputs and minimizing worst-case weight perturbations; 2, iteratively adapting the proxy model to enforce the local smoothness of loss landscape around the input x(t). This paper devises two separate loss functions for optimizing neural networks in these two steps. As a result, this method serves to avoid adversarial optimized inputs.
In the experiment section, this method is compared to the state-of-the-art methods on the Design-bench dataset. Results show that this method achieves state-of-the-art performance across a variety of tasks.

main_review: Originality:
This method involves two main steps, i.e., pre-training with weight perturbations and iterative proxy model adaptation. The first step leverages techniques from adversarial training; the second step utilizes the test-time adaptation technique. The combination of these two achieves good performance. In the related works section, literature in offline model-based optimization and regularizing smoothness of deep neural networks are summarized. There are two concerns. First, it would be helpful to give literature about test-time adaptation techniques related to the second step in this method; Second, it would be better to discuss the connections between previous methods and this method explicitly. For instance, what are the shortcomings of previous methods, and how does this method solve those shortcomings?

Quality:
Overall, this paper is technically sound:
1.	The optimization objective functions devised for each step are sensible. It is clear how those objective functions could serve to achieve the desired property.
2.	This paper’s experiments are comprehensive. The comparison between this method and other baselines on the Design-bench benchmark shows that this method could outperform previous methods across various tasks.
3.	The ablation studies also demonstrate the necessity of each component in their method.
One concern is that since this method does not outperform other methods in all tasks, some analysis on why other methods in some tasks beat this method could better explain the potential limitations of this method. This paper is a complete piece of work. It presents the full idea with comprehensive experimental results to support its claims. This paper explicitly talks about the strengths, limitations, and future works in the last section. As for the weakness, considering that other methods on two Design-bench tasks outperform this method, some discussion on possible reasons for this would be appreciated. Also, besides the lacking of actual experiments limitation, more analysis on the weakness of this method is expected.

Clarity:
This paper is clearly written. The ideas behind each component are easy to understand. One concern is the term “input-level regularization”. Some brief definitions of this term would be appreciated. For the implementation details, this paper provides comprehensive implementation details in the experiment section. Also, details about each task and how they are evaluated are provided in both the paper and its supplementary materials.

Significance:
This paper achieves good performance in the black-box objective function estimation task. Their method is easy to understand. After comparing to other methods, RoMA achieves state-of-the-art results at some tasks and overall quality. As a result, other researchers and practitioners are likely to build their work on this paper.
limitations_and_societal_impact: This paper includes comprehensive discussions on limitations, future works, and potential negative social impacts in the last section.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a gradient-based method that searches for inputs that maximizes the output of a (black-box) score function (model-based optimization).   In the analyzed scenario, the score function is an adversarially-robust pre-trained neural network, and the searching algorithm is gradient ascent (wrt to inputs) with a local regularizer in terms of the model parameters.
main_review: Pros.

- The proposed method achieves good empirical results in continuous tasks in the Design-Bench benchmark.
- The authors proposed solutions for both continuous and discrete inputs.


Cons.

My major concern is about the technical novelty and experimental setting/scope of the work.

- Wrt to novelty, the pre-training strategy follows from existing work in the context of adversarial robustness. The iterative proxy model adaptation is "gradient ascent" with a regularizer that updates the model parameters based on a hand-designed objective. I appreciate (and agree up to some extent with)  the intuitive motivation provided for this objective. However, many other regularizers can be proposed based on a similar intuition. Since this is the major technical contribution of the work, and the experimental section considers only a single benchmark with a 3 layers MLP, I believe a better theoretical motivation/justification could be provided.

- Experimental Setting/Scope: Many tasks can be formulated in the scope introduced in section 2.1 (line 74-79) some very challenging  (e.g. class-conditional image generation).  However, it is not clear why the tasks in the chosen benchmark should be considered over others. Would the proposed framework (RoMA) still be effective in more complex tasks and networks? If only such a benchmark is considered then the scope of the paper should be narrowed and motivation for the more narrowed scope should be provided.
- Writing. I consider several parts of the writing that can be improved e.g. introduction and abstract could be improved.

Questions:

- How important is regularizing the gradient norm vs keeping the output of the model close between steps? (e.g. first vs the second term in line 138) (This is not captured in figure 4).
- Why model "adaptation" instead of model adjustment? In other words, is "adaptation" the right word?  If searching starts from  x_0 ~ D, and every step is only changing x  very locally why  x_1=x_0 + delta is necessarily another distribution. 
limitations_and_societal_impact: Yes
review_title: null

Meta-review: The paper makes a somewhat incremental, yet solid and effective contribution to the area of model-based optimization by robust pre-training and a test-time robustification process. I concur with the reviewers and recommend the paper to be accepted at NeurIPS.
Decision: Accept (Poster)