Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper study the domain adaptation imitation learning. The paper proposes a method to map the state and the action of both domains into a common space. After learning this common space, the target agent can zero-shot perform the new task that only has source demonstrations. Specifically, the paper use domain adversarial learning to learn such common space of the state and the action.
strength_and_weaknesses: Strength:

The topic of cross-domain imitation learning is relatively new and has the potential to explore.

The framework of the paper (the common state and action space) is clearly stated and can inspire the following works.

The method introduces domain adaptation learning into behavior cloning, which is relatively new in the community.

Weakness:

Domain adversarial learning is a conventional method for domain adaptation when we need to learn domain-invariant representations. The paper directly uses domain adversarial learning to learn the common space of the state. It is not a very novel idea for me.

GAMA uses domain adversarial learning to learn the mapping of the state and the action between domains. The paper is more like a simple version of GAMA, as the domain discriminator of GAMA takes the triples (s, a, s') as input while the paper takes the state s as input. GAMA learns the mapping from the source state and action to the target while the paper learns a common space. Therefore, the contribution of the paper is incremental to GAMA.

Intuitively speaking, the adaptation phase might disturb the aligned space learned by the alignment phase. Because in the adaptation phase, the method will finetune the common policy for the source domain, it will cause inconsistency between the common policy and the decoder of the target domain.

The alignment scores in Table1 clearly do not favor the proposed method. If the paper would like to claim this alignment score is not a suitable metric, then it needs to propose some new metrics. 

The experimental environments have no overlap with GAMA. Therefore, it is not convincing enough that the method is better than GAMA.
clarity,_quality,_novelty_and_reproducibility: Clarity: the paper is well-written and easy to follow.

Quality: it has a self-contained framework and proposed a classic solution to the problem.

Novelty: the novelty is limited, according to the weaknesses mentioned above.

Reproducibility: good. The source code is provided.
summary_of_the_review: Overall the paper clearly states the problem and the framework, which can inspire the readers. The following works can benefit from the view of the paper, which introduces the common state and action space.

However, the novelty of the work is limited. The paper follows GAMA and combines more ideas in the domain adversarial learning. Therefore, I will weakly reject this paper and hope the author can improve the idea and the experiments.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The proposed method is as simple as aligning representations in the latent space to learn a domain-invariant policy. Conceptually that makes sense as it can generalise better in the presence of small perturbations and changes in an environment. This allows to do some domain adaptation with zero-shot learning whereby everything learnt in the source domain (environment Χ) can be transferred into the target domain (environment Υ). The results presented are fairly comprehensive and the visualisations alike, but the description of the method is not very convincing, i.e. it seems to work surprisingly well. The authors have provided the code and the corresponding datasets though.
strength_and_weaknesses: Strengths:

a) Relatively simple method, which can be extended and further improved relatively easily, especially the adversarial training part, and of course the learning part. 
b) Competitive results show promising avenues for expanding on other settings and in other environments
c) Visualisations of the latent space provide some further understanding of the underlying processes involved.

Weaknesses:

a) Background work is rather blunt. The last paragraph on page 2 is not well written. Methods' names are somewhat akwardly provided alongside the in-text citation.
b) I am not sure that the background covers all recent methods, but because I am not very well aligned with this area I am not very firm on it.
c) Further to b) more comparisons could have been added to the main text
d) Methods are not very well described in the main paper - appendix provides further context though so it somewhat compensates. It buffles me though that the method works relatively well, given what the methods employed
clarity,_quality,_novelty_and_reproducibility: The paper could benefit form some further improvements in the writing, especially on the first page, i.e. pages 1-3. It is a good paper though, with fair novel and original components, Code has been provided hence it is easily reproducible.
summary_of_the_review: The paper provides adequate experiments and visualisations to justify the advantages of the proposed method. I am somewhat hesitant due to the method description seeming too straightforward and 'simple' (not in a bad way, in that I do not mean that complex is needed) to learn representations that are transferable and generalisable. Results prove me wrong though so am erring in caution.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper presents a novel method to perform zero-shot cross-domain transfer by learning a domain-agnostic policy in the source domain that can be directly applied to the target domain. To this end, specific training methods (technical contritions) include: 
1. Align representations of states and actions from different domains to a common latent space through adversarial training. 
2. The common policy is trained to adapt to the target task in the source domain.
3. Directly apply the common policy to the target domain based on the aligned representations. 

The effectiveness of the proposed approach is verified with four Maze environments in the D4RL benchmark, which is shown to outperform other baselines especially when there are sufficient proxy tasks.
strength_and_weaknesses: Strength:
1. It is a simple yet reasonable idea for cross-domain policy transfer by aligning and training a policy in a common latent representation space shared by different domains. 
2. The authors provide both qualitative and quantitative results to show the performance of adversarial alignment. 
3. In general, the paper is well-written and easy to follow. 

Weaknesses:
1. My main concern is the novelty of the proposed approach, which seems very similar to GAMA. Both of them use an adversarial objective for state alignment in different domains so that the policy can be directly transferred to the target domain. The major difference is that in this paper, the states are aligned in a common latent space instead of a direct matching.
2. If I understand correctly, the action $a_z$ from the common policy may only contain compact information. Therefore, if action $a_d$ in the target domain is more complicated than the action in the latent space, only using $a_z$ and the domain ID might not be able to decode $a_d$. For example, $a_z$ may represent a high-level behavior such as “turn left”. However, for a robot to execute this action, it may take multiple steps based on its current domain-specific states (e.g., the pose of its own torso). In other words, the method might only work when the source and target domains have a strong relationship, because the policy in the target domain is completely determined by the common state, and does not depend on any domain-specific skills.
3. All experiments are conducted in the Maze environments in the D4RL environments, which is inadequate to support the effectiveness of the model in general control scenarios. Therefore, I would suggest validating the model on other benchmarks with a wide variety of dynamics. For a fair comparison, the authors may use the original environments of other baselines, e.g., those used in the GAMA paper.
4. Except for GAMA, the compared approaches are somewhat "out-of-date". Although I understand that this paper is focused on performing policy transfer between offline datasets, without interacting with various environments, it would be nice if the authors could include stronger baseline methods for model comparison, especially the offline RL approaches that show impressive performance on the D4RL dataset.
5. A typo: “deomnstrations” in Appendix C.2.

clarity,_quality,_novelty_and_reproducibility: 1. Clarity and quality: This paper is clearly written.
2. Novelty: Fair. The main idea is similar to the related work, GAMA, which also uses an adversarial training strategy to align states from different domains.
3. Reproducibility: Source code provided.
summary_of_the_review: Overall, I think the paper presents a simple but effective cross-domain policy transfer method. However, I am inclined to reject this paper given its technical novelty compared with GAMA and the insufficient experimental results.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper provides a novel method of cross-domain transfer with an unaligned dataset, and the proposed approach aims to acquire domain-invariant feature space and a common policy on it. And the experiments with various domain shifts show that the proposed method achieves better performance than existing methods in cross-dynamics. 
strength_and_weaknesses: Strength 
The paper proposes to use approach similar to GAN that aims to acquire domain-invariant feature space and a common policy on it. The author provided extensive ablation study, and compared the model with state of the art models, and showed the new model has better performance.
Weaknesses
The training process seems quite complicated. Combined with the instability of adversarial training, the model can be difficult to tune.
clarity,_quality,_novelty_and_reproducibility: Clarity
The paper is easy to understand, and the quality looks good.


summary_of_the_review: The paper is well written and provided extensive experiments to show the proposed method works better than baseline models.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper studies a method for zero-shot cross-domain transfer by learning a domain-agnostic policy in the source domain that can be directly applied to the target domain. At the core of the approach is aligning representations of states and actions from different domains to a common latent space through adversarial training. The proposed approach is verified with four Maze environments in the D4RL benchmark. The approach is simple and reasonable, and the improvement in performance is beneficial. While the initial reviews are controversial, the author rebuttal addressed many of the concerns, such as extra results to solve most concerns about the experiments. However, the main concern held by the two negative reviewers is the novelty of the approach: The proposed approach is too similar to GAMA -- The difference from GAMA is not substantial enough to warrant another published paper. Authors are encouraged to further enhance their paper with a more pronounced algorithm.
justification_for_why_not_higher_score: Based on the reviews, as well as the AC's take on the paper, there is an agreement that the current method is too similar to GAMA.
Decision: Reject