Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper targets goal-oriented reinforcement learning tasks with possible intervention of a virtual social partner assistant. It presents experiments for exploring object manipulation with 5 different blocks, in which an agent discovers configurations based on its own intuitive exploration, as well as guidance for other configurations from which it can further explore. The mount of intervention by the social partner is varied (from 0% to 100%), showing that 2-20% are sufficient for significant improvement over exploration without social partner.  
main_review: There are few points that I appreciate in his paper:

•	The paper investigates an important problem in robotics and AI in general, namely learning with guidance from a social partner. It relies on previous developmental psychology studies (e.g., the notion of Zone of Proximal Development by Vygotsky), and it proposes novelty in terms of a hybrid agent that both explores configurations for his own goals by interacting with its physical environment, as well as getting guiding instructions from a social partner (a social learning component).

•	The paper suggests an interesting approach for intervention by the social partner: it first let the agent to discover configurations on its own, including configurations at the boundaries of its current abilities that may serve as stepping stones for next exploration. Then the social partner suggests one if these stepping stone configurations as a goal. In this way the agent gradually discovers new configurations and extends its exploration space with only little help from the social partner.

•	The realization and implementation of the model seems to be solid and consists of a graph neural networks to represent the objects in the environment and their relations, and a knowledge graph to store previously seen semantic configurations. 
summary_of_the_review: Interesting approach based on the notion of Zone of Proximal Development for modeling goal-oriented reinforcement learning with social partner assistant
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper discusses a model for learning a task (5-block manipulation) using both "self-directed" (which the authors define as "autotelic" learning and social learning. I would say that the contribution is quite modest since the authors demonstrates the validity of their approach only through simulations of a specific game. It is difficult to evaluate the generality of the proposed approach.
main_review: 
This is an interesting paper, however, as it stands, its contribution is quite limited.

The authors also refer to literature about learning that is tangentially related to the model they use. At the end, it seems that the model itself is quite simplified and the model is only loosely connected to the concepts that are presented in the first part of this paper. It seems that the model itself is essentially a mixture of conditioned goal-oriented learning and social learning. It seems to the reviewer that the term "autotelic" might remind the reader to autonomous generation of goals, which does not seem the case for this work.

In any case, the most problematic aspect is related to the actual evaluation of the approach. In fact, the evaluation is purely experimental. This is perfectly fine, but the authors consider a rather specific example and it is rather hard to generalize for it. In particular, the part about social learning and curriculum learning appear to be effective for the task considered in the paper, but this might not be the case in general. The reviewer would suggest the authors to consider either another set of tasks and/or a more in-depth discussion of the theoretical foundations of the approach. The reviewer understands that it might not be possible in general, but, at the same time, I believe that the authors might provide the readers with a more in-depth analysis of the contribution of the self-directed learning and social learning.

Another aspect that it would be interesting to analyze in more detail is the amount of social learning that is needed and how that should be structured in order to apply the approach to other situations and tasks.

The authors should also try to improve the presentation of the proposed solution by separating the discussion of the proposed solution and the application to their specific task. For example, the authors present the problem of task decomposition, but that really refers to the problem at hand.

I found the discussion of goal-conditioned learning quite interesting, but again, it seems to me that the authors go from the general solution to their specific problem without clearly introducing it (and making clear that the problem under consideration has some specific characteristics and constraints).

The discussion of the social learning part (Section 3.3) is rather high-level (the references to the psychology literature also appear slightly out of place here). The reviewer would like to suggest adding a more detailed description of the actual social learning mechanisms.

summary_of_the_review: The authors consider a solution of a task (5-block manipulation) using "self-directed"/goal-conditioned learning and social learning. According to the reviewer, the contribution of this work is limited, since it is difficult to understand if the solution generalizes to other tasks. In fact, the evaluation is based only on that specific task.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper has explored the area of teachable autonomous agents where it highlights two contributions a design of the social interaction protocol and a learning architecture called GANSTER. Generally, the extensive experimental results have demonstrated the effectiveness of proposed method. However, there are still a few concerns. Attached below

main_review: 
1, I'm confused about the semantic knowledge graph construction. Which part of graph features that SR measures? The knowledge graph should be considered as structured knowledge where it depicts the real activities/knowledges. It seems that it exists some transition probabilities that related with the semantic graph, which shouldn't be considered as KG.

2, It is not clear in A.3 that how to alleviate the attention bias in small steps. How to quantify the small compared to others? What's the performance?

3, There should be more ablation studies over automatic decompositions. E.g. why the k=5 is the shortest path?
summary_of_the_review: Generally, this paper propose an innovative research question in teachable autotelic agents. However, some aspects of the structures, especially the semantic knowledge graph constructions.
review_title: null

Meta-review: The paper introduces GANGSTR, an agent that performs goal-directed exploration both individually and "socially", with suggestions from a partner. It builds a graph of different configurations of a 5-block manipulation domain, and navigates this graph. The theoretical motivations for this algorithm are solid, and the direction is interesting. However, the results are less than convincing. In particular, as was mentioned in the discussion, it is not clear how this algorithm would generalize beyond the very simple 5-block manipulation domain. While having a simple benchmark has the advantage that you can explore it in depth, it also might obscure problems with the algorithm, unless complemented by a set of other benchmarks. It therefore seems that the paper is not ready for publication yet.
Decision: Reject