Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a new domain adaptation SAUM method that learns a large margin classifier between different classes for cross-domain sentiment analysis. The key idea is to train a domain-agnostic embedding space based on learning a prototypical source distribution with GMM, which is then used to align domain distributions via SWD minimization. Experimental results on Amazon multi-domain review dataset empirically demonstrate that the proposed SAUM method can outperform the state-of-the-art domain adaptation baselines. Moreover, detailed theoretical and empirical analysis are provided to demonstrate the effectiveness of the proposed method.

Strengths

1.The paper is reasonably well-written and structured. 

2.The proposed SAUM method is technically sound.

3.The proposed SAUM method can outperform state-of-the-art domain adaptation methods.

4.Theoretical analysis and empirical analysis and sufficient and convinced.

Weaknesses

1.For domain adaptation in the NLP field, powerful pre-trained language models, e.g., BERT, XLNet, can overcome the domain-shift problem to some extent. Thus, the authors should be used as the base encoder for all methods and then compare the efficacy of the transfer parts instead of the simplest n-gram features.

2.The whole procedure is slightly complex. The author formulates the prototypical distribution as a GMM, which has high algorithm complexity. However, formal complexity analysis is absent. The author should provide an analysis of the time complexity and training time of the proposed SAUM method compared with other baselines. Besides, a statistically significant test is absent for performance improvements.

3.The motivation of learning a large margin between different classes is exactly discriminative learning, which is not novel when combined with domain adaptation methods and already proposed in the existing literature, e.g.,
Unified Deep Supervised Domain Adaptation and Generalization, Saeid et al., ICCV 2017. 
Contrastive Adaptation Network for Unsupervised Domain Adaptation, Kang et al.,  CVPR 2019
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation, Chen et al., AAAI 2019.

However, this paper lacks detailed discussions and comparisons with existing discriminative feature learning methods for domain adaptation.

4.The unlabeled data (2000) from the preprocessed Amazon review dataset (Blitzer version) is perfectly balanced, which is impractical in real-world applications. Since we cannot control the label distribution of unlabeled data during training, the author should also use a more convinced setting as did in
Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification, He et al., EMNLP 2018, which directly samples unlabeled data from millions of reviews.

5.The paper lacks some related work about cross-domain sentiment analysis, e.g., 
End-to-end adversarial memory network for cross-domain sentiment classification, Li et al., IJCAI 2017
Adaptive Semi-supervised Learning for Cross-domain Sentiment Classification, He et al., EMNLP 2018
Hierarchical attention transfer network for cross-domain sentiment classification, Li et al., AAAI 18


Questions:

1.Have the authors conducted the significance tests for the improvements?

2.How fast does this algorithm run or train compared with other baselines?

review_title: AnonReviewer1

final_score: rating: 5: Marginally below acceptance threshold
review_content: In this paper, the authors propose a new method to improve cross-domain sentiment analysis. They introduce large margins between classes in the source domain and it helps to reduce the possibility of misclassification on the target domain because of domain drift. After the training on the source domain, they use a Gaussian mixture model to generalize the samples and then select a subset of the drawn samples from the learned GMM for which the confidence level of the classifier is more than a predefined threshold. Therefore the drawn samples will keep larger margin between classes in the source domain. The authors provide a theoretic analysis to justify the algorithm. The experimental results demonstrate the efficacy of their method. 


Pros:
1. The authors enlarge the margin between classes on the source domain to reduce the possibility of missclassification on the target domain because of domain drift.
2. The paper analyzes their method in a theoretic way.

Cons:
1. The proposed method is very straightforward. The 'max-margin' component in the paper is implemented by the selection of samples with higher confidence of the classifier. Therefore, the performance depends on careful tuning of the threshold. A higher threshold will result in lots of samples in the source domain will be filtered out and then there is information loss during the alignment between source domain and target domain.

2. The evaluation is weak. Some experimental settings are unreasonable.

    a) The baseline methods use different dimensions of feature vectors. Some use 5000 and the other use 30000 as the vector size. For the fair comparison, both settings of 5000 and 30000 should be considered for the baseline methods. Perhaps some methods are sensitive to the variance of the vector size.

    b) How to set the threshold \tau? The author uses 0.95 without any explanation.

    c) It is not enough that only two values, i.e. 0 and 0.99 are assigned in Table2. A line should be drawn to depict the variance of performance according to different values of the threshold.

Typos:
There are some typos in the paper. The authors should proofread the draft carefully. I list some typos as follows:

'costumers' --> 'customers'

'domain shit' --> 'domain shift'

review_title: Review

final_score: rating: 5: Marginally below acceptance threshold
review_content: Summary:
The paper proposes an unsupervised domain adaptation approach for sentiment analysis. The main idea is to align the cross-domain representation with Sliced Wasserstein Distance (SWD), and train the sentiment classifier on additional pseudo-labeled target domain data. By modeling the embedding spaces as a mixture of Gaussians (GMM), with one Gaussian per sentiment class, the authors sample for pseudo-labeled data with the GMM and retain only the samples that are labeled confidently by the classifier. The authors presented a theoretical analysis leveraging on previous results for SWD for domain adaptation.  

Strengths:
- the paper presents an appealing representation learning approach to cross-domain alignment of representations. 

Weaknesses:
- By only retaining samples with high classifier confidence, the authors claim that "the margin between the clusters in the source domain increase if we use the generated pseudo-dataset for domain alignment". This seems to be an important claim in this paper, as the authors named their approach a "max-margin" approach, and they argue that it is this effect that is the reason for the good performance of their model. However, there is no proof in the paper of this claim, and I am not sure if that is true in a mathematical sense.

- As a general domain adaptation paper, the results would be more convincing with another data set. As an application paper for cross-domain sentiment analysis, the authors failed to compare against recent work such as [2], [3] and [4]. Du et al. [2] used the BERT representation, but in Table 1 in [2], the results of IATN [3] and HATN [4] seemed to outperform this current submission without using BERT.

Minor comments:
- Wrong reference for TAT, should be [1]
- Color in Figure 2 could be more distinct.
- K->E: PBLM has a higher score (87.1) but SAUM^2's 86.8 was bolded
- The paper did not explain how the standard deviations in Table 1 are obtained, for rows that have them.
- The authors should proof read the paper for typos and grammatical mistakes, e.g., "costumers", "Due to existence" missing "the", "we use the common of reducing", "our goal is mitigate" missing "to", "classier", 


[1] Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan. Transferable adversarial training: A general approach to adapting deep classifiers. ICML 2019.
[2] Chunning Du, Haifeng Sun, Jingyu Wang, Qi Qi, Jianxin Liao. Adversarial and Domain-Aware BERT for Cross-Domain Sentiment Analysis. ACL 2020.
[3] Kai Zhang, Hefu Zhang, Qi Liu, Hongke Zhao, Hengshu Zhu, Enhong Chen. Interactive Attention Transfer Network for Cross-Domain Sentiment Classiﬁcation. AAAI 2019
[4] Zheng Li, Ying Wei, Yu Zhang, Qiang Yang. Hierarchical Attention Transfer Network for Cross-Domain Sentiment Classiﬁcation. AAAI 2018

Comments after authors' rebuttal:
Thanks for addressing my comments. However, I think the current submission needs further work. 
- The authors agreed with me on my point that the "max-margin" claim might need further work, and since this is an important claim in the paper, I cannot improve my review score after the author response.
- Different data splits should not be a barrier for comparison against previous work.

review_title: Interesting paper, but there are a few issues

final_score: rating: 5: Marginally below acceptance threshold
review_content: Pros: 
The proposed SAUM2 is novel. As a contribution, the work introduced large margins between classes in a source domain. This is relevant in a cross-domain sentiment analysis problem. The result shows that the resultant model is domain-independent which is Ok for a general application. 

Cons: 
Although the SAUM2 is novel in my opinion, the algorithm seems cumbersome. It is unclear how the "domain shift" would cause performance degradation. This is so because it was not shown how Theorem 1 was an explanation of Algorithm 1. The idea looks good in my assessment but it was not shown how it would be reproducible in subsequent works. 

The following minor issues need to be revised: 
1. Abstract: The 3rd sentence needs to be revised - 'per each domain' is ambiguous. 
2. Introduction: Line 9 - 'in a other different domains' needs to be revised. In the second paragraph, the term 'domain-agnostic' needs to be defined or explained. On page 2, 'SWD' needs to be defined also or refer to its definition in Pg4, Para 1. Also on Page 2, under contributions, the phrase 'in the embedding matches this distribution' is ambiguous. 
3. Page 6, Para. 1: The section of the Appendix referred to should be specified. 
4. Page 8, Section 6.3: 'we may not matching the higher order' is ambiguous. 
review_title: Learning a Max-Margin Classifier for Cross-Domain Sentiment Analysis

Meta-review: In this paper, the authors proposed a large-margin-based domain adaptation method for cross-domain sentiment analysis.

The idea of developing a large-margin-based method for domain adaptation is not new. Though the proposed method contains some new ideas,  the difference between the proposed method and the existing large-margin based methods needs to be discussed and studied empirically.  In addition, the experimental results are not convincing: some related baselines are missing and experiments need to be conducted on more datasets. 

Though the authors did provide long responses to each reviewer, after a lot of discussions, the reviewers still find that their concerns are not well addressed. 

Therefore, this paper is not ready to be published in ICLR based on its current shape.

Decision: Reject