Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: To reduce domain shift (covariate shift due to different domain distributions), the authors propose a loss based on Equivalence Regularization and Mirrored Samples.

The main idea is to: 1) find class prototype on both source and target domain and 2) encourage each sample's relative embeddings in the source and target domains to be similar.

Experiments are carried out on 4 datasets and obtained competitive accuracy against SOTA methods.

main_review: Strengths:
- The idea of using mirrored samples to align domain embedding is quite interesting. 
- The author proposes to ER (Equivalence Regularization) to compute domain anchors as class prototypes for each domain.
- Relative vector coordinates for each sample can then be computed relative to domain anchors.
- Evaluation and ablations are conducted on 4 datasets, showing competitive accuracy against SOTA methods.

Weaknesses:
- The LNA and ER are run per-epoch, I am not convinced how they can effectively alter the embedding, if the datasets are large.
- The author seems to assume both source and target have groundtruth labels. Is this a common practice in the domain adaptation usecases? Usually the target domain label is not accessible.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper is addressing the domain adaptation problem as previous methods neglect the structural properties of the underlying distribution. This paper proposes a virtual mirror concept with Local Neighbor Approximation (LNA) and Equivalence Regularization (ER). The proposed method achieves state-of-the-art performance on several benchmarks.
main_review: 1. claim concerns:
a) For Equivalence Regularization (ER), this paper adopts clustering to obtain the class centers for target domains. However, it is critical to have reliable clusters as the anchors. How to make sure this would happen? Moreover, if the target clusters are not reliable, this will also distort the internal structure of the intrinsic underlying distribution, which is one of the main claims in the paper. The details about this concern are not clear in this paper.

2. technical detail concerns:
a) Equation (7) consists of two parts: the relation between target samples and source virtual mirrors (first loss term), and the relation between source samples and target virtual mirrors (second loss term). It would be great to discuss more these two terms. For example, which term is more critical in this problem? Why?

3. experiment concerns:
a) There are some other state-of-the-art papers missing in this paper. For example, RSDA (CVPR’ 20) also achieves competitive performance on several benchmarks (e.g., 91.1 on Office-31, 70.9 on Office-Home, and 90.5 on ImageCLEF). FixBi (CVPR’ 21) adopts similar ideas that generate virtual samples and calculate the consistency between source and target domains. Moreover, FixBi also achieves state-of-the-art performance (e.g., 91.4 on Office-31, 72.7 on Office-Home, and 87.2 on VisDA-2017). It would be better to include them in the experiment comparison (also compare the difference of approaches).

References:
1. (RSDA) Xiang Gu, et al. "Spherical Space Domain Adaptation with Robust Pseudo-label Loss", CVPR, 2020.
2. (FixBi) Jaemin Na, et al. "FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation", CVPR, 2021.

b) The proposed method contains two parts: Local Neighbor Approximation (LNA) and Equivalence Regularization (ER). The ablation studies only investigate the details about ER (in Table 2). The contribution of LNA is not very clear. It would be better to investigate LNA as well. For example, designing experiments to adopt LNA but with other types of loss functions (e.g., those in SRDC [41]).

limitations_and_societal_impact: The potential negative societal impact is not discussed in this paper.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the author proposed a sample-level alignment method for solving the covariate shift problem in unsupervised domain adaptation. They noticed that the existing methods do not work well since they neglect the bias in the datasets (e.g. different visual patterns in the same category). To fix this problem, for each sample in the source/target domain, they construct a mirror sample in the other domain by a weighted average of the top K nearest neighbors. They also derived new losses to preserve the geometric structures of latent representations in the subspace. The overall idea is straightforward but interesting. The experimental results achieve SOTA performance on public benchmarks.
main_review: technical concerns:
- In fig.1(b) one-dimension case, this example is somehow misleading. It is trying to convey the message that distribution alignment is equivalent to sample alignment. One may fail to align two distributions due to the different number of samples drawn from each distribution. This example is not convincing. 
- In fig.2, by the definition of mirror set in line 121 eq(1) (let's consider Euclidean distance), for each sample in source, its corresponding mirror set will always consist of samples close to the left boundary, for instance, blue triangles in the dashed circle. Is this true for the 2D example in fig.2?
- According to line 223-225 and Algorithm 1 in the appendix, the centers of the target are computed based on the pseudo labels that generated from the classifier. Since the classifier may produce wrong labels for target examples, these target centers can be inaccurate. The mirror losses are highly dependent on these inaccurate centers to promote the latent representations preserve geometric structures in the subspace. Does the mirror sample still a semantic counterpart? Will their labels change from one category to another? Will this introduces extra noise for supervised learning (classification task)? Is there any strategy to handle these problems? 

*Originality*: The novelty is limited. As the author claimed, the main contribution of this paper is the idea of mirror samples and mirror losses. CyCADA uses GAN to generate a 'mirror sample' in the target domain for each sample from the source domain, while in this paper mirror samples were constructed by inquiring neighbors. I am quite interested in the comparison between these two approaches. However there is no comparison experiments for these two methods.

 *Quality*: The proposed method is well motivated. Their claims have theoretical analysis and are supported with good experimental results. But most ideas rely on heuristic knowledge. Considering the aforementioned technical concerns, the proposed method for constructing mirror samples is not principled. 

*Clarity*: The paper is easy to follow. But there are typos and grammar errors, e.g., line 146 eq(3),  $\mu_c^s$ should be $\mu_c^t$ .

*Significance*: The significance of this submission is moderate.
limitations_and_societal_impact: The authors have addressed the limitations and potential negative societal impact of their work.
review_title: null

Meta-review: The paper proposes a method for unsupervised domain adaptation (UDA) by constructing "virtual mirror" samples of source domain in target domain (and vice versa) and then "aligning" the corresponding mirror pairs across domains for matching feature distributions. The method is novel (reviewer zzRU raised a concern on limited novelty by pointing out the connection with CyCADA but I think the proposed idea is sufficiently different) and shows strong empirical performance. The method has some heuristic components as pointed out by reviewers (eg, reliance on k-means to get clusters, the method to construct mirrors) however the authors have pointed out connections to optimal transport theory in their response. I suggest the authors make this more prominent in the revised version of the paper. Overall, the paper is above the acceptance threshold in my view.
Decision: Accept (Poster)