Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation
Authors: Jongbin Ryu
Jiun Bae
Jongwoo Lim
Venue:  (2020)

Abstract: In this paper, we introduce a collaborative training algorithm of balanced random forests for domain adaptation tasks which can avoid the overfitting problem. In real scenarios, most domain adaptation algorithms face the challenges from noisy, insufficient training data. Moreover in open set categorization, unknown or misaligned source and target categories adds difficulty. In such cases, conventional methods suffer from overfitting and fail to successfully transfer the knowledge of the source to the target domain. To address these issues, the following two techniques are proposed. First, we introduce the optimized decision tree construction method, in which the data at each node are split into equal sizes while maximizing the information gain. Compared to the conventional random forests, it generates larger and more balanced decision trees due to the even-split constraint, which contributes to enhanced discrimination power and reduced overfitting. Second, to tackle the domain misalignment problem, we propose the domain alignment loss which penalizes uneven splits of the source and target domain data. By collaboratively optimizing the information gain of the labeled source data as well as the entropy of unlabeled target data distributions, the proposed CoBRF algorithm achieves significantly better performance than the state-of-the-art methods. The proposed algorithm is extensively evaluated in various experimental setups in challenging domain adaptation tasks with noisy and small training data as well as open set domain adaptation problems, for two backbone networks of AlexNet and ResNet-50.

Under review as a conference paper at ICLR 2020
COLLABORATIVE TRAINING OF BALANCED RANDOM FORESTS FOR OPEN SET DOMAIN ADAPTATION
Anonymous authors Paper under double-bling review
ABSTRACT
In this paper, we introduce a collaborative training algorithm of balanced random forests for domain adaptation tasks which can avoid the overfitting problem. In real scenarios, most domain adaptation algorithms face the challenges from noisy, insuf- ficient training data. Moreover in open set categorization, unknown or misaligned source and target categories adds difficulty. In such cases, conventional methods suffer from overfitting and fail to successfully transfer the knowledge of the source to the target domain. To address these issues, the following two techniques are proposed. First, we introduce the optimized decision tree construction method, which the data each node are split into equal sizes while maximizing the information gain. Compared to the conventional random forests, it generates larger and more balanced decision trees due to the even-split constraint, which contributes to enhanced discrimination power and reduced overfitting. Second, to tackle the domain misalignment problem. we propose the domain alignment loss which pe- nalizes uneven splits of the source and target domain data. By collaboratively optimizing the information gain of the labeled source data as well as the entropy of unlabeled target data distributions, the proposed CoBRF algorithm achieves significantly better performance than the state-of-the-arr methods. The proposed algorithm is extensively evaluated in various experimental setups in challenging do- main adaptation tasks with noisy and small training data as well as open set domain adaptation problems, for two backbone networks of AlexNet and ResNet-50.
INTRODUCTION
In recent years, domain adaptation has been researched as it can help to solve major difficulties in the real world. Due to the huge overhead in labeling large-scale training data, it is desirable if an existing network can be adapted to different target domains. More importantly, it is common that the training dataset for adaptation is noisy and small, or the labels in the target domain do not match with the source or even unknown. These are inherent challenges in the domain adaptation problem as in real world it is common for the data to contain such class bias, noise and unlabeled data.
However, in practice, since the adapted networks often overfitted to the provided source data or the data distribution of the target domain is frequently quite different from the source, they do not perform well to the target domain. To properly deal with these real-world conditions with insufficient information, it is critical to learn the shared data distribution that is effective both in the source and target domain. To this end, we propose the collaborative training algorithm of balanced random forest (CoBRF) to mitigate the challenging problems such as noisy labels, lack of training data, and misaligned or unknown categories (open set categorization).
In random forests, multiple decision trees are learned by optimizing the information gain for the randomly selected subset features at each node split. Since random forests ensemble the internal decision trees, they are more robust to noise and overfitting problem than single decision trees. To improve the robustness the random forests, we take one step further by balancing the decision trees, i.e. maximizing the number of leaf nodes for the same tree depth. Our method builds more balanced decision trees by enforcing the sizes of the data in the left and right child nodes to be equal. While this split strategy is not locally optimal in terms of information gain, the resulting decision trees have far more leaf nodes, and it endows more expressive power which can be helpful in dealing with noise and unseen data classes. It also helps avoid overfitting as it prevents node committing too early

---

Under review as a conference paper at ICLR 2020
for specific pattern, or in other words, it postpones the decision as late as possible so that various discriminant information in the training data can be fully considered.
To enforce even splits while maintaining the discriminability, the CoBRF uses the hyperplanes estimated by the linear support vector machine (SVM). First, it randomly assigns the classes in the nodes to binary pseudo labels and equalizes the sizes of two pseudo classes by randomly removing data in the larger class. Then a linear classifier is found by SVM, and its hyperplane is translated until the data sizes both sides are equal. In sense, it finds the even split of the data projected onto the normal direction of the hyperplane and places the hyperplane there. The node split by the translated hyperplane simple yet effective. The ablation study in Sec. 4.2 confirms that the CoBRF boosts the performance compared to the baseline random forests.
Since the above training process only considers maximizing the information gain of labeled data in the source domain, which is referred to 'class information gain', it does not resolve the domain misalignment problem between the source and target domain. Because the target labels are not available during training, try to keep the overall distribution of the target data as close to that of the source data as possible. Since the source data are evenly split, we guide the algorithm to minimize the information gain between the source and target domain, which encourages even split of the target data also. The CoBRF combines the ideas, minimizing the 'domain information gain' between source and target data for the domain alignment while keeping the class information gain to be maximized. Note that the domain alignment term is the same as the negative information gain of the binary domain labels (source/target). Thus, the CoBRF can be seen adversarial learning, as it considers the domain information gain in an adversarial manner compared the conventional objective function of the random forest.
example of
We summarize the main contributions as three-fold.
We introduce the collaborative training algorithm based balanced random forest (CoBRF) using the discriminative and even node split function. Linear SVM with binary pseudo labeling is used to find the discriminative hyperplane and the even split ensures the decision tree to be balanced.
We also adopt the adversarial learning of domain information gain to align the source and target data distribution. To align two domains, the information gain between the source and target data is minimized, which learns the common data distribution of both the (unlabeled) target domain and the source domain data.
We perform an extensive evaluation of the domain adaptation to show the performance of the proposed method according to various challenging evaluation protocols. Specifically, it is compared to the baseline and state-of-the-art methods using noisy and small training data, and with open-set domain adaptation protocols. In both cases we observe significant performance improvements.
RELATED WORK
2.1 DOMAIN ADAPTATION
Recently adversarial learning has been one of the dominant approaches in domain adaptation with deep neural networks. The gradient reversal layer Ganin & Lempitsky (2015) introduced to train the networks that the discrimination of source and target domains is penalized. It improves the classification performance compared to the networks learned only with the source data. Tzeng et al. (2117) suggest the domain adaptation framework based on the discriminative network learning, which assigns individual weights to the source and target domains. In training the networks, they also consider the adversarial weight update align the domains. Several other domain adaptation papers in adversarial learning using conditional learning Long et al.) (2018), domain-symmant symmettii Zhang et al. (2019), and collaborative Zhang et al. (2018b) methods have been introduced. Also, in Tzeng et al. (2014) Long et al. (2015) 2016), maximum mean discrepancy (MMD)-based methods have been studied. Tzeng et al. (2014) propose the domain confusion loss to improve domain distribution alignment. Long et al. (2015) introduce the task-specific embedding and multiple kernel approach along with MMD to decrease the domain discrepancy. The residual transfer module presented in Long ett al. (2016) associates the classification ability of the source and target domain. MMD is further extended to multiple domain alignment in the joint adaptation networks (JAN) Long et al. (2017) using adversarial learning. The generative adversarial networks Radford et al. 2015) are adopted in
2

---

Under review as a conference paper at ICLR 2020
many domain adaptation methods Liu & Tuzel (2016); Sannsannkaaaan Samkaranarayaaan al. (2018) Volpi et al. (2018). CoGAN proposed by [iu & Tuzel (2016) 16) learns the joint distribution of multiple domains without corresponding image pairs. Saanaaranaraaaaan et al. (2018) propose the combined adversarial and discriminative learning method using the generator and discriminator of GAN.
2.2 EVALUATION PROTOCOLS IN DOMAIN ADAPTATION
Recently, many challenging protocols are introduced to evaluate the domain adaptation in realistic settings. Regarding domain generalization on deep neural networks [Li al. (20T8): [Balaji al. (2018), they divide multiple domain data into training and test set, then use the lcave-one-domain-ou scheme for evaluation. The domain adaptation on the partially overlapping source and target domains is presented in Zhang al. (2018a); Cao al. (2088). Multiple sources and target domains are mixed into the source or target domains in Zhao et al. (2018); Mancini et al.] (2018); Hoffman et al. (2018). The adaptable model is aimed to be learned using the distribution the multiple domains of the mixed set. Recently, several works (Saito et al. (2018); Panareda Busto & Gall (2017); Tan et al. (2019) address the open set domain adaptation. They assume that there exist unknown and partially overlapped known classes between domains. On the other hand, the domain adaptation methods under small training data (onn al. (2017) and the noisy (Sta al. (2019) are studied to address the
condition. Hong et al. (017)) use single training data per person, and [hhu et al. (2019) artificially corrupt the class labels or features of the source domain for the robustness evaluation. These protocols are challenging as they pose difficult problems of overfitting, class misalignment, noisy, lack training data, and little overlap.
2.3 RANDOM FOREST AS AN ENSEMBLE LEARNING METHOD
The ensemble of multiple learners has widely been used to avoid the overfitting problem Singh et al. (2016): Han al. (2017) 2016); Pi et al. (2016). [Sighh et al. (2016) introduce the regularization method for network learning, which works with variety set of network architectures and performs better than the existing regularization methods (i.e.dropout). Branchout Han al. (2017) is devised for layer-level regularization in visual tracking, where multiple branches of fully connected layers are randomly selected in training.
Random forest Breiman (2001) combines multiple random decision trees to build robust classifier or regressor. Random forests have been applied to many applications such as object tracking Zhang et al.) (2017), feature point detection Lindner et al. (2014), and speech recognition Black & Muthukumar (2015), to name few. However, it should be emphasized that the most important benefit is the mitigation of overfitting by ensembling multiple decision trees. As noticed in the literature Wyner ct al. (017); Gomes et al. (2017), the random forests tend not to propagate severe overfitting crror even with large number of trees.
There have been many recent works to improve the performance of random forests: Dheenadayalan let al. (016) proposes pruning nodes for efficient learning, Ristin al. (2015a) presents incremental modeling for large scale recognition, and Probst & Boulesteix (2017 investigates how to tune the number of trees. SVM Yao al. (201T): Ristin al. (2015b) random projection |Bosch al. (2007); [ossard et al. 20144) is often used as the binary classifier for better node split. Training balanced decision trees has been also an important topic Bosch et al.
et al. (2014); Yao et al. (2011): Lei et al. (2014); Ristin et al. (2015b). They split node into child nodes by the binary classifier, which is trained by evenly-divided training data in the node. We argue that training balanced random forests helps to alleviate the overfitting problem since balancing random decision trees avoids the biased distribution in the specific domain but prefers the common representation to any domains. Hence, we introduce the learning algorithm that enforces the even-split constraint by shifting the hyperplane(Sec.. I.II for balanced random forests. Although there have been studies of the balanced training of random forests, we provide elaborate training process of balanced random forests to learn common representations for the domain adaptation task. The effectiveness of the balanced random forests is shown by extensive domain adaptation experiments.
3

---

Under review as a conference paper at ICLR 2020
(h)
Figure Split examples decision tree according the split functions: (a) the conventional method chooses the split that maximizes the information gain. (b) In contrast, the proposed method additionally enforces the size of child nodes to equal, resulting in random balanced tree. Note that CoBRF has far mode nodes which improves the generalization ability for domain adaptation.
source data
popudoo label piseudo
pseude label pseudo
dt target data optimal hyperplane
(b)
(c)
Figure 2: Hyperplanes by the proposed methods. (a.b) The hyperplanes estimated by binary pseudo labels followed by translation for even split. Dotted line is the hyperplane estimated using linear SVM. The data are evenly split by the hyperplane shift (solid lines). Among these hyperplanes, the with maximum information gain is chosen: yellow hyperplane in (a). (c) In CoBRF, both the source information gain and target entropy is considered. The yellow is better in source information gain, the target data split is biased, while the blue splits the source and target evenly well.
3 PROPOSED METHOD
In this section, we first explain the limitation of the conventional random forests for the domain adaptation task, and then we introduce the even node split function in Sec 3.1 and the domain information gain for selecting the domain-aligned split function in Sec. 3.2
3.1 EVEN CONSTRAINED RANDOM FOREST LEARNING
random forest consists of multiple random decision trees, whose nodes learn a binary classifier for the randomly-selecter subset of features maximize the information gain (IG). We abuse the term node for the training data in the node interchangeably. The entropy of node n is defined as
Ec(n) = Σ
Po(n) log (pe(n)),
(I)
cEC(m)
where C(n) represents the set of classes of the data in n, and (n) the probability of class c in n (i.e., the data count of class divided by In/)). Then the information gain for node with the left
4

---

Under review as a conference paper at ICLR 2020
and right child nodes is defined as
Inu| [n|
IGe(n) = Ec(n)-
E
Ee(m)).
(e(left,
Conventionally, the simple split functions that compares only a couple of feature values are used, but recently more elaborate split functions using the linear classifiers are used Yao et al. (201] Ristin ct al. (015b). The hyperplane split function for a node n is written as
{ go left, if Wn Vx((x) < kn
(x)
(3)
go right, otherwise,
where vi (-) is the sub-feature selection function and w, and ki are the hyperplane parameters either randomly set or learned by a linear support vector machine Cortes & Vapnik (1955). The hyperplane with the largest information gain is the most discriminative classifier at the given node, but for the entire decision tree and the random forest it may not be the best option, because it causes the learned trees to be skewed and not well balanced (Fig. [aa..
We propose to add hard constraint of equal-size in splitting the node to get more balanced trees. The detailed learning process as follows. For the SVM to build a binary classifier, the classes in the node are randomly assigned to binary pseudo labels, and the training data for each class are assigned to the corresponding pseudo label. As the data sizes of the pseudo labels will be different, we randomly erase the data in the larger pseudo class to match the sizes. Then the base hyperplane (w,, and k2,) is computed to classify the binary pseudo labels.
Still, the split of the training data by the hyperplane is not equal-sized; thus update the bias k,, of the hyperplane so that the data size on each side is equal or differs at most by one (/[mt//cl- ||erggeell s 1). Geometrically this process is moving the hyperplane along the normal vector Wa. that is placed at the even split of the data projected onto the normal direction (Fig. 2a 26). Among the estimated hyperplanes from randomly selected sub-features, the one that maximizes the information gain (nc(n) is chosen as the node split function. To build a decision tree, like the conventional random forest, the node split is repeatedly applied until the maximum depth reached or too few data left in the node (Fig. lb
Inherently the proposed split method creates balanced trees, and for the same depth, the number of nodes is much larger than that of the conventional random forest. We argue that having more (leaf) nodes in the decision tree has advantages domain adaptation tasks. The conventional split function is locally optimal, but because of that, it is more susceptible to overfitting by committing too early, and eventually, it decreases the discriminative power of the entire random forest. the balanced trees, the data sizes in the leaf nodes are almost the same; thus, they represent local data distribution more faithfully. The even-size constraint can be thought of as a regularization in learning decision trees. The experimental results of the ablation study in Sec. 4.2 supports this argument.
3.2 COLLABORIVE LEARNING OF RANDOM FORESTS
Balanced data distribution is a big advantage in domain adaptation. However, as it does not use the unlabeled target data for learning. it still does not correctly align the data distribution of the source and target domain. In other words, the distribution of the target data also needs to be considered in building random forest. We propose new collaborative measure for selecting the split function that considers both the conventional IG and the domain distribution of the source and target data together. The collaborative information gain (Co-IG) is defined as
co-IG(nn) (1 A) IGe(n.) A (I))(n).
where X is a user parameter, n. is the labeled training data (in the source domain), and D is the binary domain label {source, target) representing the domain that the data belongs to. More specifically, (GD(n) is the information gain the domain distribution, when the data labels are either source or target, disregarding the classes in the source domain. The CoBRF chooses the hyperplane that maximizes co-IG when splitting the nodes.
Note that the IG on the domain distribution, (n), is subtracted in Eq 4 to ensure that we prefer even distribution of the source and target data in the child nodes. TGD(n) is minimized when both
5

---

Under review as a conference paper at ICLR 2020
BLOH&
By Tais
G &
DI
IS
BA
(a) Without domain alignment
(b) With domain alignment
Figure 3: Visualization of trees learned by the proposed methods. The white and gray circles the leaf nodes represent the source and target data fallen into the node, respectively. As a tree without domain alignment only considers the labeled source data, the target data distributions in leaf nodes are not even, whereas that with domain alignment generates more uniform splits. Refer to Sec. 3.2 and Fig. 2c
source and target data are evenly split into the children, as it maximizes the entropy of the children Thus (G) in CoBRF collaboratively enforce the even split of target data also.
(Eq. 2.
Fig. 2c illustrates the effect of co-IG compared to conventional IG in split function selection. The yellow line has higher IG as segments the source data (colored) better, but co-IG also considers the split of target data (gray). Although the blue line has lower IG than the blue, it separates the target data more evenly; thus, the blue line is chosen as the split function. The resulting decision trees by CoBRF are shown in Fig. 3
The co-IG is closely related to the adversarial learning of the network backpropagation Long et al. (2015); Ganin et al. (0166). In this framework, IGe and IGD be thought of as the classification and adversarial domain alignment, respectively. Thanks to the domain alignment term (co-IG), the CoBRF learns the robust models even with very noisy or small training data without overfitting. We validate the proposed methods from the moderate challenging condition such as 40% noise data to the very severe condition such 80% noise or merely 10% training data. Further, evaluate the open set domain adaptation, which has received attention in recent years, in the following section.
4 EXPERIMENTAL RESULTS
4.1 EXPERIMENTAL SETTING
We use three domain adaptation datasets such as Office-31 Saenko al. (2010). ImageCLEF-DA and Office-Home Venkateswara et al.) (2017). We evaluate algorithms using three challenging protocols: noisy, small training data, and weakly supervised open set domain adaptation. To train the CoBRF, we use 100 trees with maximum depth of 8. When there is training data fallen node, we prune the tree at that node. The number of randomly selected feature dimension for the SVM training is set to 250. The input feature of SVM is normalized for the stable learning of the hyperplane. We repeat the SVM training 15 times to select the optimal split in each node. Due to the space limitation, only representative results are shown in this section. Refer to the appendix for detailed information of the datasets, metric, and full experimental results.
4.2 ABLATION STUDY
We evaluate the effect of components in the CoBRF proposed in this paper. The CoBRF uses the balanced pseudo labeling ppeuudd, and the hyperplane shift (h_ (h_shift) for even data split. As binary labeling is necessary for hyperplane computation, the pseudo method uses randomly-assigned binary pscudo labels without removing the data to make label sizes equal. Therefore the four

---

Under review as a conference paper at ICLR 2020
Table 1: Ablation study of components for the split function of random forests without the domain alignment. The experiment is performed on Amazon (A), Webcam (W) and DSLR (D) domains of Office-31 with ResNet-50.
Hyperplane estimation h shift
pseudo X
mid pscudo
pscudo
mid pseudo
X
O
O
Accuracy
70.6
72.1
74.3
74.6
Table 2: The effect of À for domain alignment in the CoBRF. The experiment is performed on Office-31 with ResNet-50.
0.001
0.01
0.1
0.5
1.0
Accuracy 70.3
71.4
72.3
74.0
74.6
74.5
Table 3: The effect of the tree numbers and maximum depth training the CoBRF. The experiment performed on 40 % noisy condition of the Office-31 (W->A) with ResNet-50.
The number of trees (T)
Depth
5
10
50
100
62.2
65.4
67.5
67.7
7
60.2
63.5
67.1
67.8
58.9
63.6
67.5
68.0
9
55.6
60.0
66.2
67.6
Table 4: Performance comparison of the 60 and 80% Noisy and 10% Small training data protocol on Office-31, ImageCIEF-DA and office-Home dataset with ResNet-50.
Office-31
ImageCLEF-DD
Office-
Method
Noisy
Noisy
Noisy
Small
Small
Small
60%
80%
60%
80%
60%
80%
DAN
37.6
19.8
66.8
36.3
19.2
74.4
32.1
18.4
43.8
JAN
48.7
24.6
69.7
42.4
19.7
76.6
35.6
21.6
45.3
CDAN+E
49.8
22.0
67.5
54.5
25.0
79.6
34.0
15.1
44.2
CoBRF
65.6
44.3
74.6
67.8
32.6
79.8
56.8
46.5
51.8
combinations of (pseudo, mid pasilpsssuooo xh xh_shift are tested with Office-31. The baseline is (pseudo + no h_shiff). As shown in Table and of appendix, both balancing the pseudo labels and enforcing even splits by translating hyperplanes improve the performance.
Table shows shows the effect of the parameter À in co-IG formulation (Eq. 4 It confirms that optimizing for the cobalanced distribution helps the alignment of domain distributions. Table TableBals also represents the effect of hyperparameters in training CoBRF. The accuracy of the maximum depth 8 with 100 decision trees shows the best result.
4.3 NOISY DATA
In this experiment, the training labels of the specified portion of the source domain are randomly changed for the noise condition, which is also referred to the label corruption in Shu et al. (2019). Corruption levels are set to 40, 60 and 80% of the source domain (refer to the appendix for full experimental results).
We conduct noisy conditions for the Office-31, ImageCLEF-D and Office-Hoone datasets in Table and We test DAN Long al. (2015), JAN Long al. (2017), and CDAN+E Long et al. 2018
7

---

Under review as a conference paper at ICLR 2020
Table 5: Performance comparison of the 40% Noisy protocol on Office-31 with ResNet-50.
Domain adaptation
Method
A-+
A-+
D-A
D-W W
W-+A
W-D
Average
RTN Long
76.1
64.6
49.0
71.7
56.2
82.7
66.7
ADDA Tzong of 2017
61.2
61.5
45.5
65.1
49.2
74.7
59.5
Mentor Hong
75.0
74.4
43.2
70.6
54.2
85.9
67.2
TCL Shu 2019
83.3
82.0
60.5
77.2
65.7
90.8
76.6
CoBRF
81.9
82.1
65.4
81.1
68.0
92.8
78.5
Table 6: Performance comparison of the OpenSet1 protocol on the Office-31 dataset with
AlexNet Krizhevsky et al. (2012).
Method
A-+
A-
D-
D
W-A
W-D
Average
OSVM
59.6
57.1
14.3
44.1
13.0
62.5
40.6
ATI-A + OSVM
72.0
65.3
66.4
82.2
71.6
92.7
75.0
Saito et al. (2018)
76.6
74.9
62.5
94.4
81.4
96.8
81.1
CoBRF
86.0
80.5
73.0
94.5
69.4
94.6
83.0
Table 7: Performance comparison of the OpenSet2 protocol on the Office-31 dataset with ResNet-50. Results of CoBRF* are from a more challenging setup. Refer Sec. 4.5
Method
A D
+
W
++
W
Average
JAN Long el
JANLLonger
2017
65.5
63.8
74.7
68.0
ATI-semi sem Pananeda
72.0
73.4
77.8
74.7
CDA Tan 2019
75.2
77.1
88.1
80.1
CoBRF
82.3
83.1
92.9
86.1
CoBRF*
82.0
81.2
89.7
84.3
algorithms on the same noisy condition for comparison. Table 5 shows the result of 40% noisy training data for Office-31 with ResNet-50 al. (2016). The proposed CoBRF outperforms all other algorithms in average accuracy. The result confirms that the CoBRF improves the performance in most settings, and interestingly, Table/[show shows the more severe the noise is, the larger the performance improvement gets.
4.4 SMALL TRAINING DATA
In this experiment, we use only 10% training samples to evaluate the performance against overfit- ting. We perform the experiments Office-31, -ofice-Hoom, and ImageCLEF-Da. datasets with 50sNet-50. The result of Table shows the CoBRF achieves favorable performance compared to the other algorithms. Full experimental results presented in the appendix.
4.5 OPEN SET EXPERIMENTS
We perform two open set evaluation protocols proposed in Saito et al. (2018): Tan et al. (2019).
OpenSetl: The first open set protocol Saito al. (2018) uses 11 classes (10 known and unknown) of the Office-31 dataset. The labels from to of both source and target domains are marked as the known class, and all data with label 21-31 1 in the target domain According to /Saito et al. (2018) the unknown class the source data is not used in training, and the unknown class of the target data is classified by thresholding the class probability. The thresholding value is set to 0.3. Shows the result of CoBRF as well as the state-of-the of-the-rt methods. The
used one unknown class.
DAN, JAN: http:///gihhbb.com//mm////eeee CDAN+E: http:///tthhbbccmmmmm//AAAAA
8

---

Under review as a conference paper at ICLR 2020
CoBRF achieves the best performance among all algorithms on Office-31. It also demonstrates the effectiveness of the proposed method under the challenging adaptation condition.
Recently, another open set protocol is proposed in Tan al. (2019). which partially overlapping known classes between the source and target domain. Each domain has known-and- common classes, 5 noown-but-diffreen classes, and unknown class for all other training data, thus in total there are 15 known and unknown classes. First, according to Tan et al. (2019), 3 samples per class per domain and 9 samples in the unknown class per domain are used in training. Hence the total number of training samples is (3 samples x 10 classes/domain + samples_ in__nknown) x domains 78. All other algorithms and CoBRF results in Table 7 are using this protocol.
OpenSet2:
Additionally, we evaluate more challenging setup, where the training data are sampled regardless of the domain, i.e., the data in common classes (including unknown) are merged before being sampled. In this case, 3 samples in CoBRF rows are acquired this setup. We confirm that CoBRF works well compared to state-of-the-ar methods under the OpenSet2 and more challenging condition.
15 classes 9 samples_i _i___iinniia jn_unknown= 54 in total are used. The results
CONCLUSION
We propose novel cobalanced random forest (CoBRF) algorithm for challenging conditions and open set protocols. The CoBRF enhances the discriminative ability of the random forest by building balanced decision trees by the even split. The proposed CoBRF algorithm also employs the adversarial learning for domain alignment and benefits the effectiveness against the overfitting to the labeled source data. We extensively evaluate the proposed algorithms using challenging experimental protocols and demonstrate its superior performance over the baseline and state-of-the-art methods.
REFERENCES
Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain gener- alization using meta-regularization. In Neural Information Processing Systems, pp. 998-1008 2018.
Alan W Black and Prasanna Kumar Muthukumar. Random forests for statistical speech synthesis. In Conference of the International Speech Communication Association, 2015.
Anna Bosch, Andrew Zisserman, and Xavier Munoz. Image classification using random forests and ferns. IEEE International Conference on Computer Vision, 2007.
Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101-mining discriminative compo- nents with random forests. European Conference on Computer Vision, pp. 446-461. Springer, 2014.
Leo Breiman. Random forests. Machine Learning, 45(1):5-32, 2001.
Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Michael I Jordan. Partial transfer learning with selective adversarial networks. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 2724-2732, 2018.
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 2033):273-277, 1995.
Kumar Dheenadayalan, G Srinivasaraghavan, and VN Muralidhara. Pruning a random forest by learning a learning algorithm. In International Conference Machine Learning, pp. 516-529. Springer, 2016.
Yaroslav Ganin and Victor Lempitsky. Unsupervised domain adaptation by backpropagation. In International Conference on Machine Learning. pp. 1180-1189, 2015.
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. aomain-adversaree training of neural networks. Journal of Machine Learning Research, 17(1 17(1):2096 1:209--2000, 2016.

---

Under review as a conference paper at ICLR 2020
Heitor M Gomes, Albert Bifet, Jesse Read, Jean Paul Barddal, Fabrício Enembreck, Bernhard Pfharinger, Geoff Holmes, and Talel Abdessalem. Adaptive random forests for evolving data stream classification. Machine Learning, 106(9-10):1499-11 1191144999444,,,,, 2017.
Bohyung Han, Jack Sim, and Hartwig Adam. Branchout: Regularization for online ensemble tracking with convolutional neural networks. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3356-3365, 2017.
Shizhong Han, Zibo Meng, Ahmed- Khan, and Yan Tong. Incremental boosting convolutional neural network for facial action unit recognition. In Neural Information Processing Systems, pp. 109-117, 2016.
Kaiming He. Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.
Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. Algorithms and theory for multiple-souree adaptation. Neural Information Processing Systems, pp. 8246-8256, 2018.
Sungeun Hong, Woobin Im, Jongbin Ryu, and Hyun S Yang. Sspp-dan: Deep domain adaptation network for face recognition with single sample per person. Arxiv, 2017.
Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. Mentornet: Learning data- driven curriculum for very deep neural networks on corrupted labels. In International Conference on Machine Learning, pp. 2309-2318. 2018.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolu- tional neural networks. In Neural Information Processing Systems, 2012.
Yanqiang Lei, Guoping Qiu, Ligang Zheng, and Jiwu Huang. Fast ncar-duplicate image detection using uniform randomized trees. ACM Transactions Multimedia Computing, Communications, and Applications, 10(4):35, 2014.
Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 5400-5409, 2018.
Claudia Lindner, Paul A Bromiley, Mircea C Ionita, and Tim F Cootes. Robust and accurate shape model matching using random forest regression-voting. 3729)1162211874 2014.
Ming-Yt Yu Liu and Oncel Tuzel. Coupled generative adversarial networks. In Neural Information Processing Systems, pp. 469-477, 2016
Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan. Learning transferable features with deep adaptation networks. International Conference on Machine Learning, pp. 97-105. JMLR. org, 2015.
Mingsheng Long. Han Zhu, Jianmin Wang, and Michael Jordan. Unsupervised domain adaptation with residual transfer networks. In Neural Information Processing Systems, pp. 136-144, 2016.
Mingsheng Long, Han Zhu, Jianmin Wang, and Michael Jordan. Deep transfer learning with joint adaptation networks. In International Conference Machine Learning, pp. 2208-2211 JMLR. org, 2017.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael Jordan. Conditional adversarial domain adaptation. In Neural Information Processing Systems, pp. 1640-1650, 2018.
Massimiliano Mancini, Lorenzo Porzi, Samuel Rota Bulò, Barbara Caputo, and Elisa Ricci. Boosting domain adaptation by discovering latent domains. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 3771-3780. 2018.
Pau Panareda Busto and Juergen Gall. Open set domain adaptation. In IEEE International Conference Computer Vision pp. 754-763, 2017.
10

---

Under review as a conference paper at ICLR 2020
Te Pi, Xi Li, Zhongfei Zhang, Deyu Meng, Fei Wu, Jun Xiao, and Yueting Zhuang. Self sacf-paced boost learning for classification. In International Joint Conferences on Artificial Intelligence, pp. 1932-1938, 2016.
Philipp Probst and Anne-Laure Boulesteix. To tune or not to tune the number of trees in random forest. Journal of Machine Learning Research, 18:181-1, 2017.
Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. Arxiv, 2015.
Marko Ristin, Matthieu Guillaumin, Juergen Gall, and Lue Van Gool. Incremental learning of random forests for large-scale image classification. 3803)-490-503, 2015a.
Marko Ristin, Matthieu Guillaumin, Juergen Gall, and Luc Van Gool. Incremental learning of random forests for large-scale image classification. IEEE transactions on pattern analysis and machine intelligence, 38(3)4490-533, 2015b.
Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell. Adapting visual category models to new domains. In European Conference Computer Vision. Springer, 2010.
Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, and Tatsuya Harada. Open set domain adaptation by backpropagation. In European Conference on Computer Vision, pp. 153-168, 2018.
Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa. Generate to adapt: Aligning domains using generative adversarial networks. In IEEE Conference Computer Vision and Pattern Recognition, pp. 8503-8512, 2018.
Yang Shu, Zhangjie Cao, Mingsheng Long, and Jianmin Wang. Transferable curriculum for weakly- supervised domain adaptation. In AAAI Conference on Artificial Intelligence, 2019.
Saurabh Singh, Derek Hoiem, and David Forsyth. Swapout: Learning ensemble of deep architec- tures. In Neural Information Processing Systems, pp. 28-36, 2016.
Shuhan Tan, Jiening Jiao, and Wei-Shi Zheng. Weakly supervised open-set domain adaptation by dual-domain collaboration. Arxiv, 2019.
Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. Arxiv, 2014.
Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. Adversarial discriminative domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 7167-7176, 2017.
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, 2017.
Riccardo Volpi, Pietro Morerio, Silvio Savarese, and Vittorio Murino. Adversarial feature augmenta- tion for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 5495-5504, 2018.
Abraham J Wyner, Matthew Olson, Justin Bleich, and David Mease. Explaining the success of adaboost and random forests as interpolating classifiers. The Journal of Machine Learning Research, 18(1):1558-159) 2017.
Bangpeng Yao, Aditya Khosla, and Li Fei-Fei. Combining randomization and discrimination for fine- grained image categorization. In IEEE Conference Computer Vision and Pattern Recognition, 2011.
Jing Zhang, Zewei Ding, Wanging Li, and Philip Ogunbona. Importance weighted adversarial nets for partial domain adaptation. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 8156-8164, 201 8a.
11

---

Under review as a conference paper at ICLR 2020
Le Zhang, Jagannadan Varadarajan, Ponnuthurai Nagaratnam Suganthan, Narendra Ahuja, and Pierre Moulin. Robust visual tracking using oblique random forests. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 5589955598 2017.
Weichen Zhang, Wanli Ouyang, Wen Li, and Dong Xu. Collaborative and adversarial network for unsupervised domain adaptation. In IEEE Conference on Computer Vision and Puttern Recognition, pp. 3801-3809, 2018b.
Yabin Zhang, Hui Tang, Kui Jia, and Mingkui Tan. Domain-symmetric networks for adversarial domain adaptation. Arxiv, 2019.
Han Zhao, Shanghang Zhang, Guanhang Wu, José MF Moura, Joao P Costeira, and Geoffrey J Gordon. Adversarial multiple source domain adaptation. In Neural Information Processing 2018.
Systems, pp.
12

---

Under review as a conference paper at ICLR 2020
Appendix
A DATASET AND METRIC
We use three domain adaptation datasets for the experiments.
Office-31 is the domain adaptation dataset, which consists of three domains, Amazon (A), Webeam (W) and DSLR (D). The dataset has 4,652 images with categories, and we evaluate all combinations of the domains transfer following the previous work.
SrEgeCLEF-Da includes 12 classes each domain with 50 images per category per domain. The three domains of Cal6ech-256 (C), ImageNet ILSVRC 2012(I) and Pascal VOC 2012 (P) are used for evaluation.
Office-Home with 15,500 images and 65 categories is larger dataset than office-31. We use four domains Artistic (A), Clip Art (C), Product (P) and Real-World (R) for evaluation.
We follows the standard metric for the experiments. The averaged accuracy over the classes Saito ct al. (2018); Tan et al. 2019) used to OpenSetl and OpenSet2, and for other experiments the classification accuracy used as the metric.
B ABLATION STUDY
The full tables of ablation study in the manuscript are presented.
Table 8: Ablation study of components in CoBRE.
Hyperplane estimation
Domain adaptation
h shift
A-D
A-W
D-
+A
W-
Average
pseudo
X
73.3
73,6
57.3
81.4
53.7
84.4
70,6
pie_uo
X
74.4
75.0
58.3
81.2
57.2
86.8
72.1
pseudo
O
76.4
78.1
58.1
82.8
61.0
89.4
74.3
mid pseudo
76.1
77.3
59.0
83.4
62.0
89.6
74.6
Table 9: The effect of X in CoBRF. The experiment is performed on Office-31 with ResNet-50.
Domain adaptation
À
A-D
A-W
D
D-W
W-A
W-D
Average
76.8
75.3
51.7
75.2
55.1
87.7
70.3
0.001
76.2
76.3
51.7
78.3
57.3
88.4
71.4
0.01
75.9
76.5
52.9
80.8
58.7
88.9
72.3
0.1
77.4
76.9
57.6
81.7
61.0
89.2
74.0
0.5
76.1
77.3
59.0
83.4
62.0
89.6
74.6
1.0
76.0
77.3
59.9
83.6
60.9
89.2
74.5
13

---

Under review as a conference paper at ICLR 2020
C
NOISY DATA
The full tables of noisy protocol in the manuscript are presented.
Table 10: Performance comparison of the 60 and 80% Noisy protocol on Office-31 with ResNet-50.
A-+D
A-
D-*A
D
D
W-A
W-+
Average
Method
60
80
60
80
60
80
60
80
60
80
60
80
60
80
DAN
44.8
23.2
39.5
22.1
17.5
12.0
49.4
22.8
20.4
13.5
53,9
25.2
37.6
19.8
JAN
62.2
29.7
61.6
32.6
28.4
17.4
56.3
26.1
28.0
14.1
55.4
27.8
48.7
24.6
CDAN+E
56.7
24.4
59.0
26.0
34.4
13.3
55.1
24.0
35.8
17.4
57.6
27.3
49.8
22.0
CoBRF
78.0
68.6
79.5
68.5
49.7
28.4
58.4
29.3
58.2
35.7
69.8
35.5
65.6
44.3
Table 11: Performance comparison of the 60 and 80% Noisy protocol on ImagoCLEF-DA with ResNet-50.
C++1
C-P
I-C
I-P
P-C
P-I
Average
Method
60
80
60
80
60
80
60
80
60
80
60
80
60
80
DAN
33.3
19.2
25.8
15.6
42.9
29.1
35.1
17.3
40.2
16.3
40.8
17.9
36.3
19.2
JAN
39.6
18.9
33.6
15.3
49.2
28.9
38.6
17.3
50.8
19.7
42.5
18.1
42.4
19.7
CDAN+E
57.2
24.2
41.8
19.0
68.1
34.0
52.3
23.1
52.9
26.3
55.0
23.7
54.5
25.0
CoBRF 72.4
34.4
60.1
30.2
73.8
39.8
61.3
31.7
71.4
31.0
67.8
28.7
67.8
32.6
Table 12: Performance comparison of the 60 and 80% Noisy protocol on Office- Home with ResNet-
50.
Mothod
Ar-SBt
Ar-*Rw
Cl-+Ar
CI-RRw
Pr-
Pr-+CI
Pr-sRw
Rw-+CI
Rw
Average
60
60
60
60
60
60
60
60
80
60
80
80
80
80
80
DAN
11.0 5.2 362
18.6
43.3
22.8
28.4
18.2
42.9
22,9
41.5
23.5
29.8
159
21.3
13.6
23.6
13.1 38.4 23.6
15.2 9.6
54.3 33.1
32.1
18.4
JAN
21.9
10.6
406
212
44.5
22.
3166
18.1
43.1
24.7
432
25.4
31.0
174
23.3
148
49.3
29.7
16.8
24.
26.1
160
56.1
54.2
35.6
21.6
CDANHE
22.2 9.8 41.7
20.7
49.4
13.9
24.9
11.4
38.5
17.1
37.9
16.6
10.
17.4
33.3
16.2
10.3
$2.6
27.6
34,0
15.1
CoBRE 38.1 26.2 96.3 38.9 64.6 44.9 51.5
41.2
59.1
47.2
61.0 50.2 53.1
46.9
43.1 36.8 71.3 63.2
62.7 $7.0
46.7 40.5 73.7
68.2
568 46.5
D SMALL TRAINING DATA
The full tables of small training data protocol in the manuscript are presented.
Table 13: Performance comparison of the 10% Small training sample protocol on Office-31 with ResNet-50.
Method
A
A-
D-
D-
W
W-
Average
DAN
69.7
69.8
48.7
74.2
53.7
83.9
66.8
JAN
74.7
76.7
49.8
76.6
55.4
85.2
69.7
CDAN+E
77.8
76.6
42.3
75.0
49.5
83.6
67.5
CoBRF
76.1
77.3
59.0
83.4
62.0
89.6
74.6
14

---

Under review as a conference paper at ICLR 2020
Table 14: Performance comparison of the 10% Small training sample protocol on ImagcCLEF-DA with ResNet-50.
Method
C-+1
C-P
I-C
I-P
P-C
P-1
Average
DAN
79.4
65.9
86.0
67.5
74.3
73.2
74.4
JAN
81.3
71.0
89.5
69.0
73.8
75.3
76.6
CDAN+E
87.8
70.8
91.3
71.3
79.8
76.9
79.6
CoBRF
83.2
72.2
90.1
74.1
80.7
78.6
79.8
Table 15: Performance comparison of the 10% Small training sample protocol Office-Home with ResNet-50.
Method
A-C
A-P
A-R C-A
C-P
C-R
P-A
P-C
P-R
R-A
R-C
R-+P Average
DAN
22.3
38.5
47.9
35.3
44,9
46.4
44.3
32.5
62.8
54.1
33.9
63.2
43.8
JAN
22.3
38.2
49.3
48.3
65.1
55.4
32.1
45.3
CDAN+E
21.5
35.1
46.2
36.0
45.3
46.2
45.0
33.4
63.0
56.6
33.2
69.7
44.2
CoBRF
32.2
49.3
55.3
45,0
53.5
55.3
52.4
39.8
67.9
59.0
42,6
69.8
51.8
15


Reviews:
[{"review_id": "ryg6fy_ptB", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a new target objects for training random forests that has better generalizability across domains. The authors demonstrated that the proposed method outperforms existing adversarial learning based domain adaptation methods.\n\n\nStrength\n\nThe paper is clearly-written. The two objectives(balanced split and common split distribution between source and target domain) are well motivated and explained in the paper.\n\nThe authors show that empirically the proposed method outperform several existing adversarial learning based domain adaptation methods.\n\n\nWeakness\n\nOne of the main draw back of the method is that it relies on the features extracted from existing pre-trained neural networks, and cannot be used to update the representation of the neural networks. While the adversarial learning based method could do end to end training.\n\nIt would be great if the authors could clarify the setup of the baseline methods(e.g. Whether the baseline methods also take benefit of imagenet dataset, and is trained end to end).\n\nWhat will happen if you do not have the imagenet models and have to train all the models from scratch?\n\nOverall I think it is a borderline paper that might be interesting to some audiences in the conference.\n", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "S1l_sCFXcS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes an approach to building random forests that are\nbalanced in such a way as to facilitate domain adaptation. The authors\npropose to split nodes not only based on the Information Gain, but\nalso so that the sizes of each set passed to left and right children\nare equal. Another extension to the standard random forest training\nprocedure is the use of a collaborative term subtracted from the\ninformation gain over the source domain. This term encourages\nalignment of the source and target domains in the leaves of trees in\nthe forest. Experimental results are given on a range of standard\nand open-set domain adaptation datasets.\n\nThe paper has a number of issues:\n\n1. There are some problems with clarity, and the English is somewhat rough\n   throughout. These problems are not terribly distracting, but the\n   manuscript could use more polish.\n2. I don't see a detailed discussion anywhere about the\n   hyperparameters used for fitting the random forests. How many trees\n   are used? What is the max depth? These parameters should be\n   discussed and included in the ablations in order to appreciate the\n   complexity/performance tradeoffs.\n\nThis paper has some interesting ideas in it, and the experimental\nresults are excellent. I would encourage the authors to move salient\nmaterial from the supplementary material to the main article and to\nprovide a more thorough discussion of the complexity of the models\n(the structural parameters of the trees/forests).", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "H1gbR6xTtB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper introduces a method for domain adaptation, where each domain has noisy examples. Their method is based on a decision tree in which the data at each node are split into equal sizes while maximizing the\ninformation gain.  They also proposed a way to reduce domain alignment. Their method is tested on several noisy domain adaptation settings and performs better than other baseline methods. \n\nPros:\nTheir idea to utilize a decision tree for domain adaptation sounds novel. \nExperiments indicate the effectiveness of their method.\n\nCons:\nThis paper is not well-written and has many unclear parts. \n1, The presentation of the problem set is unclear throughout this paper. In the abstract, they mentioned that they tackle the situation where both source and target domains contain noisy examples. However, they did not define the exact problem setting in any section. I could not understand what kind of problem setting motivated their method, which makes it hard to understand their method. \n2, How they actually optimized the model is also unclear. From Eq 1~4, it is hard to grasp how they trained the model. \n3, In open-set domain adaptation, simply minimizing domain-distance can harm the performance. How does the method avoid this issue? It was also unclear. \n4, Experimental setting seems to be wrong and unclear. In Openset1, they say that \"The labels from 1 to 10 of both source and target domains are marked as the known class, and all data with label 11∼20 in the source domain and label 21∼31 in the\ntarget domain are used as one unknown class\". However, Saito et al. (2018) used 21-31 classes in the target domain as one unknown class. In addition, \"According to Saito et al. (2018) the target data of the unknown class is not used in training, \", they used the 21-31 classes for training in an unsupervised way. How is this method used to detect unknown class? Is there any threshold value set for it?\n5, The experimental setting is unclear. In 4.4, \", we use only 10% of training samples\", does it mean 10 % training source examples or target examples? This setting is also unclear. \n\nFrom the cons written above, this paper has too many unclear parts in the experiments and method section. I cannot say the result is reproducible given the content of the paper and the result is a reliable one. They need to present more carefully designed experiments. \n", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes new target objectives for training random forests for better cross-domain generalizability. 

As reviewers mentioned, I think the idea of using random forests for domain adaptation is novel and interesting, while the proposed method has potential especially in the noisy settings. However, I think the paper can be much improved and is not ready to publish due to the following reviewers' comments:

- This paper is not well-written and has too many unclear parts in the experiments and method section. The results are not guaranteed to be reproducible given the content of the paper. Also, the organization of the paper could be improved.

- The open-set domain adaptation setting requires more elaboration. More carefully designed experiments should be presented. 

- It remains unclear how the feature extractors can be trained or fine-tuned in the DNN + tree architecture. Applying trees to high-dimensional features sacrifices the interpretability of the tree models, hampering the practical value of the approach.

Hence, I recommend rejection.
Decision: Reject