Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper proposed a skill transfer method which exploits learned semantic correspondence in order to transfer skills from source domains to novel target domains. Although the means of executing actions to achieve skills might be very different in terms of low level control, the proposed method can transfer the skill from source domain to the target domain with high performance. The method was verified in two simulation environments and with real human video demonstration data. The method was compared against SOTA methods that do not use semantic information and shown to exhibit superior performance.

strengths_and_weaknesses: - Especially, generalization using real human video demonstrations and automatic extraction of skill labels are very impressive.
- The paper is well-written but a heavy read. A lot of details were only provided in the Appendix, which makes reading the paper challenging. Additionally, dropping the notation for the source/target domain reduces readability.
- The assumption of predefined skill annotations is a very strong assumption. Yet, if the demonstrations/dataset is from robot motions, it is acceptable because these movements are programmed/learned for some purpose, which might be used as the label of the corresponding segment. Additionally, the authors convincingly showed the use of action recognition models in automatic labeling. What is left is the assumption of having "the same set of semantic skills" in between different (source and target) domains. The authors might discuss what happens if the semantic skill set is not exactly the same but similar.
- Related to the previous comment, in order to match the semantic skills, one way might be to focus on the changes the robot creates in its environment. Even though the changes are not exactly the same with different embodiments and arrangements, still the relative changes created on objects by the end effector of the robot would be similar across different domains/environments.
- The paper was shown to have superior performance against baselines that do not use semantic information. I am curious how it would perform if such information is included in other skill transfer methods, for example, in ACNMP which is a demo+RL method that can efficiently transfer skills between different embodiments.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The approach is novel, the method is sound, and the methods are verified rigorously in simulation experiments and real data. The paper works on a well-motivated problem. It opens a possibility for a scale-up if we can indeed extract information from demonstrations available in other domains.
issues: In Eq. 4. I did not understand how aligning p_t(k | s^T) and p_s(k | s^S) helps transfer from source to target since s^T and s^S are not paired. In a sense, we align these two distributions but not necessarily align if they output the same k for semantically same s^T and s^S. I think Eq. 5 partially helps for this issue by also considering multiple timesteps, but I am not sure if this is generic. A clarification helps.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: This paper presents an approach for imitation across cross-domain based on semantic skills.
The approach can utilize demonstrations across substantially differing domains to accelerate the learning of long-horizon tasks.  This approach can be used to leverage human video demonstrations for teaching new tasks to the robot  while being robust to noise in training and demonstration data.
The approach is tested across two simulated maze and kitchen environments as well as from real world videos of human to a simulated robot. The results show that the approach can accelerate learning from cross domain demonstrations.
strengths_and_weaknesses: Strengths:
The paper is very well written and organized. The motivation and problem setup is very clear.
The approach presents a promising way to use large scale video datasets for behavior learning in robotics.

Weakness:
The following points could be addressed:
What are the computational challenge of learning cross domain skill? e.g. Time for  episodes?
How would it transfer to real robot manipulation from a human video demonstration?

Update:
The complexity of algorithm is well addressed.

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The paper has interesting ideas for scalable and generalizable robot learning with relevant experimental results.
I would recommend accept.
issues: Analysis of time complexity  and insights about human video to real world manipulation.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes a method to leverage demonstrations collected in a different instance of an environment, to accelerate the learning process of the same tasks in a new instace (e.g. two different kitchens). They do so by assuming the existence of different datasets, one with demonstrations of the full task at a high-level, in a different and non-accessible environment, and a task agnostic dataset collected on the current environment, where the agent can interact with the objects.
The general method is interesting and effective, although there are many moving parts and I wonder if it couldn't be made simpler.
strengths_and_weaknesses: Stengths:
- The paper tackles an interesting problem: bootstraping reinforcement learning with demonstrations of a long horizon tasks that come from a different instance of the same type of environment. Results show that the method can indeed accelerate the learning phase, and surpasses the performance of many baselines.
- The experiments section is well designed and shows several ablations that help convincing the reader of the benefits of the method.
- Using a real world video of a person solving those tasks can indeed show the potential of such methods to scale to larger datasets composed of humans solving tasks. This data is plentiful online and could possibly enable exponential growth of the capabilities of the robot.
Weaknesses:
- The method is convoluted and has many moving parts. There are several losses that are based on regularising the policy with a prior learned on the existing datasets. These parts could possibly be swapped for normal supervised learning, making it simpler and possibly without sacrificing performance.
- It is unclear if some baselines fail at a high-level or low-level. In line 244 the authors specify that a baseline mostly fails because the low level policies fail, but the plan is correct. This should be better investigated, as learning low level policies should be easier, and generally not the main point of the paper.
- The task is relatively linear in execution, and therefore, as they also show, simply learning or emulating the sequence of high level action can achieve good performance. As specified before, it is unclear if such benchmarks fail at a high-level (difficult to solve) or low-level (easy to solve). Figure 7 shows that if some skills are missing from the demonstration, these techniques indeed fail. However, I believe it would still be possible to learn a good planner with a larger dataset.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I suggest to accept the paper in the current form: the idea is interesting, and the problem they tackle is fundamental for the robot learning community. However, I would suggest the authors to minimise the number of moving parts of their algorithm, that can dramatically hurt reproducibility, and show, possibly with more complex robot environments, the benefits of this method for high-level planning than a BC-like planner
issues: As described above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper focuses on imitation learning of high-level semantic skills where the source and target domain are different. The paper propose "STAR", a model dealing with cross-domain knowledge of long-horizon skills, based on an assumption that the low-level skills are already learned from the given dataset in both source and target domains. The work outperforms several baseline methods in the aspect of learning speed, result metrics, and no need for demonstrations in the target domain.
strengths_and_weaknesses: Pros:
- The motivation and problems to be solved are well explained.
- Good presentation of related works and their limitations.
- The assumptions in the problem formulation are properly made and do not violate common sense.
- Table 1 in the Appendix helps understand different models and their training objectives.
- The experiment results look very interesting and promising, especially the last experiment on learning semantic skills from human demonstrations in a video. The current model solved some of the drawbacks of the related works. 
- The limitation section is well written, letting me know that the authors share the same concern as mine, i.e. the existence of pre-defined skills and corresponding data in all the domains.

Cons:
- The approach and model in section 4.1 are not well formulated. From line 113 to line 139, I could not follow the techniques and causality the authors wanted to explain.
- The role of the latent representation z in the current model is not clear. In line 129, the author gives an example saying "z capture the robot hand orientation", but z should have some other meaning, such as a high-level execution description of the task. 
- Why a tuple (k,z) can represent a sequence of H steps?
- Model and math lack details and the explanation is not sufficient to help understand the model, e.g., for Eq. 1, I do not understand how did the authors reach this learning objective, and the content from line 135 to line 139 cannot help me understand it, neither the reasons of "capturing the discrete aspects of manipulation skills". 
- The model formulation since Section 4.2 becomes easier to understand. However, the entire model seems very complicated. It has a lot of submodules that need rich experience to get trained, which decreases my confidence in reproducing the result by other researchers and allowing the people beyond the authors' group to extend this work. 

robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: I feel very difficult to determine if this paper should be recommended or not. Thus, I would like to hear other reviewers' opinions. My difficulties come from: (1) I am not familiar with the related works. (2) Most of the presentation of the paper seems correct and the experiments look promising, but the entire model is very complicated. In my opinion, it will be quite hard to be used or extended by other researchers. So I do not feel confident to evaluate the contribution and influence of this paper. 
issues: I can capture the main idea and the majority of the model formation of this paper. Due to the lack of knowledge of the related works, however, I cannot easily judge the correctness of the technical details. 
But I think the content between line 127 and line 139 can be improved, as I got confused when I read it. 

- Give more details to help the understand Eq. 1 and the reasons to support the claims from line 136 to line 139. If the proofs are from related works, please explicitly pointed out where (paper, section, eq, etc.) can I find them.

- Line 129, gives a more proper description of the latent representation z instead of "z captures the robot hand orientation", which may also explain why "A single tuple (k, z) represents a sequence of H steps"

- Why can this inference network help learn the low-level policy network?

Other question:
- What is the difference between the proposed method and a method which is predicting the order of executing skills? E.g. in Figure 4 middle, given some demonstrations, the prediction result is "2, 3, 1, 4", and then the robot just executes the basic skills following the order "skill2"->"skill3"->"skill1"->"skill4", as the low-level skills are already learned. 

Typo:
- Table 1 in Appendix, the training obj. should be Eq(1), instead of Eq.(2).
review_title: null

Meta-review: metareview: The paper is well written and organized, motivation and problem setup clearly stated, tackling an interesting and fundamental problem of bootstrapping RL with information from a different problem instance, and showing promising results, especially when learning from human videos.

However, there are several concerns raised that should be considered and clarified. Mainly regarding the explanation of the complex method, impact of assumptions, and baseline comparisons. See the following proposed action items based on the provided reviews (more details in the corresponding reviews).

Proposal of action items:
- Clarifying the notion of ‘domain’ as a different kitchen is still a kitchen domain, a different domain would be something like a workshop or factory. Also, e.g., L91 source and target ‘environments’ are used, and then switching to ‘domains’. Both wordings appear in several section of the paper and should be clarified.
- Discussing the assumption that semantic skill sets have to be the same for transfer
- Consider a baseline method that also includes semantic information
- Consider an analysis of time and computation complexity
- Clarifying the failures of the baselines (high- vs. low-level)
- Clarifying the approach and modelling descriptions
- Discussing the consequence of the complex model design involving many different parts, pretraining etc. Maybe having additional ablation studies to investigate their influence.

------

Following the consensus recommendations of the reviewers, and enhanced by the additional updates and clarifications of the paper during the rebuttal, my recommendation is to accept the paper.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)