Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper describes an approach to domain adaptation that uses
self-supervised losses to encourage source/target domain alignment for
unsupervised domain adaptation. The authors propose to use four
self-supervised tasks (variants of tasks used in the self-supervised
representation learning for object recognition literature) that are
used with a combined loss including unlabeled source and target
training samples. The authors also propose an alignment heuristic for
guiding early stopping. Experimental results on a standard battery of
domain adaptation problems are given, plus some intriguing baseline
results for semantic segmentation.

The paper is written very well and the technical development and
motivations for each decision are well discussed and argued.

1. The experimental evaluation is a bit limited as the object
   recognition datasets are a bit limited. Results on Office or
   Office-Home would be nice.

2. Using location classification for semantic segmentation seems
   intuitively to be encouraging the network to learn coarse spatial
   priors (which should be invariant across the two domains). Have you
   looked at how alignment is actually happening? More qualitative
   analysis in this direction would be useful to appreciate the
   proposed approach.

3. Related to the previous point, it would be interesting to see how
   semgmentations in the unsupervised domain gradually change and
   improve with increasing alignment.

In summary: the ideas are simple, intuitive, and well-explained -- I
think the results reported would be easy to reproduce with minimal
head scratching. The experiments are interesting and not overstated.

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper introduces an unsupervised domain adaptation method that uses self-supervised tasks to bring the two different domains closer together. It runs experiments on some classic benchmarks.

My score for this paper is weakly rejected because 

(1) the concept of self-supervision is not first proposed by this paper. The proposed method is not novel. It introduces three simple self-supervision tasks: flip, rotation and location, and the performance is not better than previous results such as DIRT-T; 

(2) there are 7 benchmarks in Table2, but only 2 of 7 has result on R+L+F. In the paper, it mentioned because the result is not better, but the author should still provide them. 

(3) it emphasizes the contribution of encouraging more study of self-supervision for unsupervised domain adaptation. It doesn’t provide any way for how to design self-supervision task or whether more tasks is better. I think it is an interesting paper, but not enough as a conference paper, maybe a workshop paper. 

(4) there are some classic unsupervised domain adaption benchmarks like Office Dataset, and Bing-Caltech dataset, why not run the method on them?

(5) In ICCV 2019, there is a paper "S4L: Self-Supervised Semi-Supervised Learning". The proposed method is almost same. I think the difference is this paper changes the setting and considers the unsupervised data as target domain and supervised data as source domain. 
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper presents a novel unsupervised domain adaptation framework for neural networks. Similarly to existing approaches, it performs adaptation by aligning representations of the source and the target domains. The main difference is that this alignment is achieved not through explicitly minimizing some distribution discrepancy (this usually leads to challenging minimax optimization problems). Instead, the authors propose to use a battery of auxiliary self-supervised learning (SSL) tasks for both domains simultaneously. Each task is meant to align the source and the target representations along a direction of variation relevant to that task. Assuming that the battery is diverse enough, optimizing the representation for all the tasks leads to matching of the distributions. 

Pros:
+ The paper is well-written and easy to read.
+ I like the simplicity of the idea and the fact that it achieves competitive performance without any adversarial learning (which may be very tricky to deal with).
+ The paper presents a reasonable procedure for hyper-parameter tuning and early stopping which seems to work well in practice.

Cons:
- The paper is purely practical with no theory backing the approach. As a result, the discussion of guarantees and limitations is quite brief.
- It’s unclear how easy it is to come up with a reasonable set of SSL tasks for a particular pair of domains. It seems that it may become a serious problem when the method is applied to something other than benchmarks. Table 2 reveals that there is no consistent improvement over the existing approaches which suggests that the chosen battery of SSL tasks is not universal (as the authors themselves admit). On a related note, it’s a bit disappointing that the authors mention SVHN results as a failure case but never provide a way to address the issue.
- It would be nice to some results for the Office dataset for completeness. The authors could use a pre-trained network as a starting points just like it’s done in other papers. According to the last paragraph of Section 6 this experiment should be feasible.

Notes/questions:
* Table 2, last column: The performance of DIRT-T seems to be better than that of the proposed method and yet the latter is highlighted and not the former.

Overall, I think it’s a good paper presenting a thought-provoking idea. In my opinion, the weakest point of the work is the lack of any (neither principled nor practical) guidance as to how to choose the set of self-supervised tasks. Despite this I feel that this submission should be accepted but at the same time I’m curious to see what the authors have to say regarding the concerns I raised in my review.
review_title: Official Blind Review #2

Meta-review: Thanks for your detailed replies to the reviewers, which helped us a lot to clarify several issues.
Although the paper discusses an interesting topic and contains potentially interesting idea, its novelty is limited.
Given the high competition of ICLR2020, this paper is still below the bar unfortunately.
Decision: Reject