Venue: ICLR 2023 Workshop RRL

Reviews:

final_score: rating: 4: Good paper, strong accept
review_content: I found the paper to be clear, concise and introduces a well designed system to integrate instruction manuals into the RL setting as applied to the widely-used atari set of environments. While the authors are not the first to integrate instructions and RL, this is to my knowledge the most complex task to which it has been applied and the authors provide a full end-to-end system. 

This is not to say that this is not without some caveats that the authors discuss. The biggest difficulties seem to be in that of object detection, detection of interaction between objects, and the definitions of these interactions in terms of the manual's text. The first two points can hopefully be improved with stronger vision modules, however it would be great to see more discussion of what it would mean for more general interactions, i.e. more so than "What happens when the player hit a <object>?" and how this would impact the visual detection of such interaction. And to be clear these comments are in reference to future work, as the work presented by the authors is already a fantastic step in that direction.

Overall I found this to be a great paper to read and should be accepted to the workshop!

minor question: It was not clear if the multiple object tracker (L307) was used in conjunction with SPACE/CLIP or as a replacement to it. It reads to me as this was used in conjunction, but this could be clarified.
review_title: Good paper that should generate great discussion

final_score: rating: 4: Good paper, strong accept
review_content: This is a neat paper that aims to leverage the information contained in instruction manuals to improve RL agent performance on Atari games. To do so, it combines existing models (e.g. RoBERTa fine-tuned on SQUAD, Macaw, SPACE) with manually curated prompts, and object detection (on the game Skiing) or state position from RAM states (for other games), to provide auxiliary rewards based on collision-detection and its impact on performance (according to manuals).
This results in improvemed agent performance in the games considered.

One thing I'd recommend the authors consider for future work is how this idea could be extended to things other than Atari games and/or situations where you don't require human curation for the questions/prompts.

I think this is a nice paper and fits well with the workshop.
review_title: Neat paper combining manual "reading" and RL

Decision: Accept(Oral)