From: NeurIPS.cc/2022/Workshop/Offline_RL/Paper24/Reviewer_Zhti
title: Official Review
review: This paper addresses a very interesting analysis around the idea of zero-shot RL: a generalist agent that can solve any RL task given a reward-free learning phase. It studies approaches based on successor features (very good coverage) and forward-backward representations, and presents an excellent analysis of their benefits and limitations.

The appendix is a minefield of great resources for background understanding, derivations, and connections, and paints a complete picture. The empirical evidence seems solid and makes a good case for FB and Laplacian eigenfunctions.
rating: 8: Top 50% of accepted papers, clear accept
confidence: 5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NeurIPS.cc/2022/Workshop/Offline_RL/Paper24/Reviewer_5go2
title: Extensive empirical analysis of unsupervised RL
review: The paper presents an extensive analysis of successor features and forward-backward features methods on the unsupervised reinforcement learning benchmark (URLB). Specifically, the zero-shot RL setting is evaluated where a new task needs to be solved given a reward function without any learning or planning. Several methods for learning the single state features for SF are evaluated. While many methods performance variance across tasks, forward-backward and SF with laplacian features perform best overall. Overall, the paper presents an extensive analysis of a very relevant problem; these results comprise a valuable set of baselines and a benchmark for future work.

While the paper is generally well-written and the review of related work is otherwise good, I find the discussion of model-based approaches very shallow. The stated definition of 'zero-shot reinforcement learning' excludes any planning-based methods, despite this being a large and impactful class of RL method (PETS, PlaNet, muZero). Even if it is not the focus of the work, a more complete discussion of related model-based and offline RL approaches would improve the paper.
rating: 7: Good paper, accept
confidence: 5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature