Venue: ICLR 2023 Workshop RRL

Reviews:

final_score: rating: 4: Good paper, strong accept
review_content: 
This paper proposes a new approach to RL, which is based on the idea of structural neighbours. The authors show that this approach can be used to improve the performance of a variety of RL algorithms, including TD(0), Dyna and Dueling DQN. The paper is well-written and easy to follow. The experiments are well-designed and the results are convincing. Overall, I think this is a good paper and I recommend it for publication.


Pros:
- New framework for RL
- Convincing numerical results

Cons:
- The method relies on the concept of a "structural transformation function", which involves an environment-specific transformation of the state space. This is a bit of a limitation, as it requires the user to specify this transformation function.

## Comments:

### 1. Introduction

- "RL agents can solve just about any environment they are presented with". In my opinion, this is a bit too strong. I would suggest rephrasing it to something like "RL agents can solve a wide range of environments they are presented with".

- "It despite these successes, this approach only scales so far: it is simply not feasible to solve every problem from scratch by brute force.". This sentence is a bit confusing, as it seems to suggest that RL is a brute force approach. I would suggest rephrasing it. 

#### 4. Structural Reinforcement Learning

- After equation (5), "...where η ∈ [0, 1] $\eta \in [0,1]$ is a mixing factor such that $\eta = 0$ corresponds to no structural updates, and $\eta = 1$ corresponds to no temporal updates"- Is this correct? I would have expected the other way around.
review_title: Original idea, backed up by experiments.

Decision: Accept(Poster)