Venue: NeurIPS 2022 Workshop Offline_RL

Reviews:

final_score: rating: 7: Good paper, accept
review_content: ## Summary
This paper focus on the problem of model-based offline reinforcement learning, where the authors propose a new algorithm that utilizes an ensemble of models to improvement the policy. Specifically, the authors propose to first construct a collection of models where the loss is no more than $\alpha$ larger than the loss of the best model, and then update the policy via a minimax objective by improving the worse case expected return for models in the model collection. The authors show theoretically that the proposed algorithm leads to performance improvement guarantee when the data coverage is sufficient. 


## Review

Overall I think the paper proposes an interesting idea of using adversarial model objectives for provable policy improvement, so I recommend acceptance of the paper. The algorithm formulation is simply and clear, so it is intuitive to see why the proposed method enjoys the improvement guarantee. However, it would be great if the authors could also provide some empirical results, since the proposed method seems easy to implement in practice.


review_title: Review

Decision: Accept