Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper proposes an approach to building random forests that are
balanced in such a way as to facilitate domain adaptation. The authors
propose to split nodes not only based on the Information Gain, but
also so that the sizes of each set passed to left and right children
are equal. Another extension to the standard random forest training
procedure is the use of a collaborative term subtracted from the
information gain over the source domain. This term encourages
alignment of the source and target domains in the leaves of trees in
the forest. Experimental results are given on a range of standard
and open-set domain adaptation datasets.

The paper has a number of issues:

1. There are some problems with clarity, and the English is somewhat rough
   throughout. These problems are not terribly distracting, but the
   manuscript could use more polish.
2. I don't see a detailed discussion anywhere about the
   hyperparameters used for fitting the random forests. How many trees
   are used? What is the max depth? These parameters should be
   discussed and included in the ablations in order to appreciate the
   complexity/performance tradeoffs.

This paper has some interesting ideas in it, and the experimental
results are excellent. I would encourage the authors to move salient
material from the supplementary material to the main article and to
provide a more thorough discussion of the complexity of the models
(the structural parameters of the trees/forests).
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper proposes a new target objects for training random forests that has better generalizability across domains. The authors demonstrated that the proposed method outperforms existing adversarial learning based domain adaptation methods.


Strength

The paper is clearly-written. The two objectives(balanced split and common split distribution between source and target domain) are well motivated and explained in the paper.

The authors show that empirically the proposed method outperform several existing adversarial learning based domain adaptation methods.


Weakness

One of the main draw back of the method is that it relies on the features extracted from existing pre-trained neural networks, and cannot be used to update the representation of the neural networks. While the adversarial learning based method could do end to end training.

It would be great if the authors could clarify the setup of the baseline methods(e.g. Whether the baseline methods also take benefit of imagenet dataset, and is trained end to end).

What will happen if you do not have the imagenet models and have to train all the models from scratch?

Overall I think it is a borderline paper that might be interesting to some audiences in the conference.

review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: Summary:
This paper introduces a method for domain adaptation, where each domain has noisy examples. Their method is based on a decision tree in which the data at each node are split into equal sizes while maximizing the
information gain.  They also proposed a way to reduce domain alignment. Their method is tested on several noisy domain adaptation settings and performs better than other baseline methods. 

Pros:
Their idea to utilize a decision tree for domain adaptation sounds novel. 
Experiments indicate the effectiveness of their method.

Cons:
This paper is not well-written and has many unclear parts. 
1, The presentation of the problem set is unclear throughout this paper. In the abstract, they mentioned that they tackle the situation where both source and target domains contain noisy examples. However, they did not define the exact problem setting in any section. I could not understand what kind of problem setting motivated their method, which makes it hard to understand their method. 
2, How they actually optimized the model is also unclear. From Eq 1~4, it is hard to grasp how they trained the model. 
3, In open-set domain adaptation, simply minimizing domain-distance can harm the performance. How does the method avoid this issue? It was also unclear. 
4, Experimental setting seems to be wrong and unclear. In Openset1, they say that "The labels from 1 to 10 of both source and target domains are marked as the known class, and all data with label 11∼20 in the source domain and label 21∼31 in the
target domain are used as one unknown class". However, Saito et al. (2018) used 21-31 classes in the target domain as one unknown class. In addition, "According to Saito et al. (2018) the target data of the unknown class is not used in training, ", they used the 21-31 classes for training in an unsupervised way. How is this method used to detect unknown class? Is there any threshold value set for it?
5, The experimental setting is unclear. In 4.4, ", we use only 10% of training samples", does it mean 10 % training source examples or target examples? This setting is also unclear. 

From the cons written above, this paper has too many unclear parts in the experiments and method section. I cannot say the result is reproducible given the content of the paper and the result is a reliable one. They need to present more carefully designed experiments. 

review_title: Official Blind Review #3

Meta-review: This paper proposes new target objectives for training random forests for better cross-domain generalizability. 

As reviewers mentioned, I think the idea of using random forests for domain adaptation is novel and interesting, while the proposed method has potential especially in the noisy settings. However, I think the paper can be much improved and is not ready to publish due to the following reviewers' comments:

- This paper is not well-written and has too many unclear parts in the experiments and method section. The results are not guaranteed to be reproducible given the content of the paper. Also, the organization of the paper could be improved.

- The open-set domain adaptation setting requires more elaboration. More carefully designed experiments should be presented. 

- It remains unclear how the feature extractors can be trained or fine-tuned in the DNN + tree architecture. Applying trees to high-dimensional features sacrifices the interpretability of the tree models, hampering the practical value of the approach.

Hence, I recommend rejection.
Decision: Reject