Venue: NeurIPS 2022 Workshop Offline_RL

Reviews:

final_score: rating: 7: Good paper, accept
review_content: This paper proposes a new pipeline for offline RL tuning. It mainly consists of two steps:
(1). learn policies with various degrees of conservatism in the train stage, which the level of conservatism is a parameter in the policy. The loss is a weighted combination of reward, and a penalty which penalizes the distance to the behavior policy. The weight is the "parameter" that controls the conservatism.
(2). In the test time, the practitioners could start with BC and continuously increasing the weight to be less conservatism, through trail and error, to find the optimal weight.

The idea is interesting and the experiments over industrial benchmark datasets show that there is a concave shape of the weight vs. performance curve. It would be great if the authors could think about more intelligent approach in the second stage "searching" period, such as how to find the optimal weight using the least samples, and it would be great to say more about the IB dataset?
review_title: Nice paper about user interactive offline RL, which learns various policies (differed in their proximity to the behavior policy) at once in training time, and selectively tuning in the test time.

Decision: Accept