Domain Generalization by Learning and Removing Domain-specific Features
Authors: Yu Ding
Lei Wang
Bin Liang
Shuming Liang
Yang Wang
Fang Chen
Venue: NeurIPS 2022 Accept (2022)
Keywords: Domain Generalization
Domain-invariant Features
Domain-specific Features
Transfer Learning

Abstract: Deep Neural Networks (DNNs) suffer from domain shift when the test dataset follows a distribution different from the training dataset. Domain generalization aims to tackle this issue by learning a model that can generalize to unseen domains. In this paper, we propose a new approach that aims to explicitly remove domain-specific features for domain generalization. Following this approach, we propose a novel framework called Learning and Removing Domain-specific features for Generalization (LRDG) that learns a domain-invariant model by tactically removing domain-specific features from the input images. Specifically, we design a classifier to effectively learn the domain-specific features for each source domain, respectively. We then develop an encoder-decoder network to map each input image into a new image space where the learned domain-specific features are removed. With the images output by the encoder-decoder network, another classifier is designed to learn the domain-invariant features to conduct image classification. Extensive experiments demonstrate that our framework achieves superior performance compared with state-of-the-art methods.

Domain Generalization by Learning and Removing Domain-specific Features
Yu Ding University of Wollongong yd6240uowmail.c edw.aaaaliuuuueaee..m.emmm au
Lei Wang University of Wollongong
Bin Liang
University of Technology Sydney Bin LiangOuts edu.
leiw@uow. edu.
Shuming Liang
Yang Wang
University of Technology Sydney Shuming Lianguuts....... edu.au
University of Technology Sydney Yang. Wang@uts. edu. au
Fang Chen University of Technology Sydney Fang. Chen@uts. edu. au
Abstract
Deep Neural Networks (DNNs) suffer from domain shift when the test dataset follows distribution different from the training dataset. Domain generalization aims tackle this issue by learning a model that can generalize unseen do- mains. In this paper, we propose new approach that aims to explicitly remove domain-specifie features for domain generalization. Following this approach, we propose a novel framework called Learning and Removing Domain-specifh fea- tures for Generalization (LRDG) that learns a domain domain-invarinn invariant model by tactically removing
features from the input images. Specifically, sign a classifier to effectively learn the Nomain-sppccite features for each source domain, respectively. We then develop encoder-decode input image into a new image space where the learned domain -specifc features network, another invariant features to conduct image classification. Extensive experiments demonstrate that framework achieves superior performance compared with state-of-the-art methods. Code available at https: ftttp:://gitub..com http://tttwbboooollllaaaannngaoo
de-
network to map each
are removed. With the images output by the encoder-decode classifier is designed to learn the domain-invariant
Introduction
Deep Neural Networks (DNNs) have achieved great performance in computer vision tasks (26]. However, the performance would drop the test dataset follows a distribution different from the training dataset. This issue is also known domain shift 39.. Recent research has found that DNNs tend to learn decision rules differently from humans [17 16] For example, in ImageNet based 37 image classification tasks, Convolutional Neural Networks (CNNs) tend to learn local textures to discriminate objects, while we humans could use the knowledge of global object shapes as cues. The features learned by the DNNs may only belong specific domains and not generalized for other domains. For example, in real-world photos, objects belonging to the same category have similar textures, but in sketches (27). objects are only drawn by lines and contain texture information. For a CNN that uses textures to discriminate objects in the photos, poor performance can be expected when it is applied to the sketches. This situation calls for DNNs that can learn features invariant across domains instead of learning features that are domain-specific.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

In this paper, we focus on the research topic of domain generalization and follow the multiple source domain generalization setting in the literature. Its goal is to train a model that can perform well on unseen domains. In this setting, we can access multiple labeled source domains and one or more unlabeled target domains. All the source and target domains share the same label space. During the training process, the source domains are available but the target domains are unseen. The target domains are only provided in the test phase.
One typical approach to domain generalization is to learn Nomain-invariant representations across domains [8] 30 42
14 45 31 [3]. This approach is based on the assumption that each domain has its domain- specific features and that all domains share domain invariant features. For example, textures are domain- domain-specific features for the photos but shapes are both photos and sketches. Previous works propose methods that seek to distill the domain-invariaan features. Although demonstrating promising performance, these methods do not clearly inform the deep neural networks that the domain-specifer features shall be effectively removed. Instead, it is only hoped that they would be removed through achieving the final goal of learning the domain-invariant features. The lack of this clear guidance to the network may affect its learning efficacy. In this paper, we propose a new approach that aims to explicitly remove the domain-specinn features in order to achieve domain generalization. As indicated above, CNNs tend to learn the domain-specific features rather than the -inaaiinniivriantt features for classification. To prevent this from taking place, we actively remove the Nomain-speciife features and guide the CNNs to learn the domain invariant features for classification. Following this approach, we propose a novel framework: Learning and Removing Dommain-ppeciiic features for Generalization (LRDG).
features for
Our framework consists of domain-specife classifiers, an encoder-decoder network, and a domain- invariant classifier. The training process of our framework includes two steps. In the first step, each domain- -specii classifier is designed to effectively learn the domain-specific features from one source domain. Specifically, domain-specific classifier is designed to discriminate the images across different classes within one particular source domain. At the same time, this classifier required to be unable to discriminate the images across different classes within any other source domain. Each source domain therefore corresponds to one domain- sppciifc classifier under this design. In the second step, the encoder-decoder network maps the input images into the domain-specde Somain-specifie fcatures learned above are to be removed from the input images by utilizing the domain-specific classifiers. Different from the first step, each domain-specifie classifier here is unable to discriminate the mapped images across different classes within the corresponding source domain. The mapped images are expected contain much fewer domain-specifie features compared with the original input images. The domain-invann invariant classifier is then appended to the encoder-decoder network and trained with the mapped images. By this design, the encoder-decoder network actively removes the domain-specific features and the domain-invariant classifier will be better guided to learn the domain-invariant features. Once trained, the encoder-decoder network and the classifier will be used for the classification of the unseen target domains.
image space where
It is worth noting that our framework is different from the data augmentation based methods for 34 46 77. Our framework aims to remove the domain-specine features from the input images while the data augmentation based methods generate various images with novel domain-specific features. Besides, our framework just maps the input images into a new image space and does not augment them to enlarge the training dataset.
domain generalization 43
We demonstrate the effectiveness of our framework with experiments on three benchmarks in domain generalization. Our framework consistently achieves state-of-thh-art performance. We also experimentally verify that our framework effectively reduces the distribution difference among the source and target domains according to the generalization risk bound in the literature (2).
2 Proposed framework
Assuming that we are given N source domains D, = DD,
D2 D....... D.N which follow different distributions. For each domain (dataset), DJ = [(x), 15) where n, is the number of samples in
Dj and (xx, vf) ) is the data-label pair for the jith sample in the ith domain. Following the literature, we assume that all source and target domains share the same label space. The goal of domain generalization is to use these source domains D, to learn a model for the unseen target domain D,.

---

(a)
x
Le
x
Lu
x Images from source domain D
Lu
x
2
Le
FN
Lu
F3: Domain-specific Classifier
for source domain D
Lu
Lu
Lc
by Uncertainty Loss
freeze
freeze
freeze
Le Classification Loss
(b)
LR
F
Ly
M Encoder-decoder Network
F Domain-invariant Classifier
Ly
z Mapped Images of x
M
Ly
LR Reconstruction Loss
Le
x Images from target domain D,
prediction
M
An overview of the proposed framework LRDG. (a) The domain-specifer classifier F trained with the classification loss Le on the source domain D; and the uncertainty loss Lu on the remaining source domains. After training, the weights of all the domain-specifte classifiers are frozen. (b) The encoder-decoder network M is trained with the reconstruction loss LR and the uncertainty loss Lu through the domain-spectee sppeific classifiers. Meanwhile, the Nomain-invarianl classifier F is trained with the classification loss Le on the mapped images. (c) the testing phase, the encoder-decoder network M and the domain iivariinnvvriant classifier F are used for classification the target domain D,.
Figure 1:
Our work is inspired by recent work [32), where it uses a "lens" network (i.e. image-to-image translation network) to remove "shortcuts" (low-level visual features that CNN can quickly learn, such as watermarks and color aberrations) from input images in a sulf-super learning task. Differently, our work focuses on removing the domain-specific features from the input images for domain generalization task. We use an encoder-decoder network similar to the "lens" network, but we design different method to leverage the encoder-decoder network to remove the domain-specific features. In this section, we illustrate frramwork in detail. We also provide theoretical analysis for our framework. Fig. gives an overview of the entire framework.
2.1 Learning -ppaii- features
Our framework starts by training N individual domain-specific classifiers Fs (F1, "2....,Fy! in which the classifier F. is designed to only the domain-specific features from the source domain Dř to discriminate images. The domain-specific classifiers Fs should not use the domain- invariant features as cues. In other words, F; is expected be able effectively discriminate images across different classes within D; but should be difficult for F; to discriminate images across different classes within any other domains. Domains excluding D; are used to maximize the classification uncertainty or adversarially increase the difficulty of classification for F,. The classification performance of F, on the domains excluding D should be similar random guess.
Specifically, the classifier F; is trained by minimizing a classification loss C on D
arg min E Dz DD
(1)
8,
and maximizing an uncertainty loss cfs on the remaining domains (D.....,D', D....,Di-,, Di Di+1
DN
arg max EDD.D, # [x (F.
3

---

where 0g denotes the parameters of the classifier F, Lc and Ly are the classification loss function and the uncertainty loss function, respectively. We use the cross-entropy loss as the classification loss. For the uncertainty loss, since we aim to make the prediction similar to a random guess, we use entropy loss
Lo (F(
= Eple ply = [x); [m1
(.)) 0.)) logp(y = ([F((())
(3)
where the number of classes and p(y = (x (x;;; 0:)) denotes the probability of x belonging to class I. Least likely loss [22] is alternative to the entropy loss. The classifier first predicts an image and obtains the probabilities of all the classes. The class with the lowest probability is called the least likely class. This image is assigned with label of this class. Then train the classifier to predict the least likely class. The least likely loss is
Lu (F.(x()):))) 0.)) Lo LC (Px,(x;; 0.). Jf), where 0z arg
arg min p(yl 0,)).
(4)
However, experiments show that the entropy loss can better achieve the classification randomness than the least likely loss, so we use the entropy loss as the default uncertainty loss.
After training, freeze the parameters 0 of these domain-specific classifiers Fs and these classifiers to learn domain-invariant features.
2.2 Removing domain-speciffe features
To remove the domain-spectio features learned by the domain-specific classifiers, we utilize an encoder-decoder network M that maps the images into new image space 2. The output images are fed into the commain-specifc classifiers Fs and a new domain-invariant classifier F.
Unlike the training of the domain-specific classifier F; where the source domain Då is used for minimizing the classification loss, the contrary, the source domain D' in this step used to maximize the uncertainty loss C./.
arg max EDj
(F
(5)
The parameters 0g of F; are frozen and the parameters MM of the encoder-decoder network M are trained. Maximizing the uncertainty loss forces the output image z; M (x.) to contain fewer domain-specific features than the input images. In doing so, the encoder-decoder network can remove the domain-specific features in the input images and retain domain-invarian features in the output images
To maintain the overall similarity between the input and output images, we add reconstruction loss C for the encoder-dccoder network,
arg min EE Di~D. E D (L 0M), x()l
8M
where LR is the reconstruction loss function. We use pixel-wise l2 loss as the default reconstruction loss for its simplicity and reasonably good performance. Other reconstruction losses could also be employed, such as pixel-wise 4 loss and perceptual losses [24]. Detailed discussion available in the supplementary material.
We then train the Nomain-invariaa classifier F by minimizing the classification loss LFM on the output images of all the source domains,
arg min EDY~D, Doo 00.0p
where OF are the parameters of the domain invariant classifier F. This classification loss L also updates the encoder-decoder network to prevent the encoder-decoder network from losing the domain-invarian features due to the uncertainty loss. The uncertainty loss also has the potential to remove the domain-invariant features if it difficult to separate the domain-specific features from the domain-invarian features.

---

Overall, when training the Somain-specifie classifiers we optimize
L1 = - + X and when learning the domain innarintn features, optimize
(8)
L2 = C + x228 + A3C1
(9)
where Å1 X2 and λ hyperparameters that control the relative weight of these losses.
For convenience, we denote the encoder-decoder network M and the domain-invariant classifier F as model. In the testing phase, the on the target domain Dr.
model is used for classification
2.3 Explanation of LRDG with respect to existing theory
We first introduce the generalization risk bound for domain generalization the effectiieness of our framework with respect to this.
2 and then further explain
Theoretically, corresponding task for domain is defined as a deterministic true labeling function ʃ. where f' X' + y. X' and ) are the input space and the label space, respectively. We denote the space of the candidate hypothesis as H. where hypothesis h: X' + y. The risk the hypothesis on domain D is defined as
R = Ex-D(((((x)) f(x)))
(10)
where L : 3' x y - R4 measures the difference between the hypothesis and the true labeling function.
Following (2). for the source domains D D....., DN we define the convex hull Ag the source domains as a set of mixture source distributions: As (D D(.) Σ (.) 0 - mi S 1. LE = 1}. We also define Dr As as the closest domain to the target domain D,. D. is given by arg min, ......N dx (D,, >m miD where dxl:; is H-divergence [25] that quantifies the distribution difference of two domains. We use the following generalization risk bound [2 for the target domain D,
Theorem (Generalization risk bound (2)) Given the previous setting, the following inequality holds for the risk [h],,, Vh E H. for any domain DE
7+ RI S Σ Ri +
2 + A
(11)
where y = dD, c (D D2) and X the minimum sum of the risks achieved by some h E H on Dr and Dr. Y measures the distribution difference between the source domains and the target domain. is the maximum pairwise H-divergence among source domains.
Theorem shows that the upper bound for the target domain depends on y and c. We show that our framework could lower the value of this generalization risk bound for given domain generalization task. Recall that our encoder-decoder network maps the input images into new image space. We denote the mapped source domains as (D), D...... ... DN } and the mapped target domain as D.. With the domain-specific classifiers, many domain-speont -spccffcc features and the features the mapped source domains tend to more mapped source domains {D., D2 ......... D.N) would have smaller distribution difference than the raw source domains, i.e. dæ (Di, Díl du(D),, DJJ, indicating that in Eq. 11 would probably be reduced. After removing the Nomain-speciff features for each source domain, the mapped target domain D3 would be closer to the mapped source domains, so our framework could also be likely reduce y in Eq. [I Concerning Theorem[] [[] these changes provide a principled explanation and warrant to the effectiveness of the proposed framework. We will demonstrate these changes in the experiment section (Sec. 3.3
removed from the source domains
As a result, the
3 Experiments
We evaluate our framework on three benchmark datasets and compare the performance with previous methods. After that, we study the domain divergence among the source and target domains.
5

---

3.1 Datasets and settings
Datasets. We evaluate our framework on three object recognition datasets for domain generalization.
PACS [27 contains four domains: Photo (P), Art Painting (A), Cartoon (C) and Sketch (S) with
each domain covering seven categories including dog, elephant. giraffe, guitar, horse, house, and person. VLCS [39 also has four domains: PASCAL VOC 2007 (V), LabelMe (L), Caltech (C) and
Sun (S). The images belong to five categories of bird, chair, car, dog, and person. Office-Home [40]
has images from 65 categories over four domains including Art (A), Clipart (C), Product (P), and Real- Woal-World (R). For each dataset, following the literature, the experimental protocol is to consider three domains the source domains and the remaining
the target domain.
Networks and loss functions.
use U-net [36] for the encoder-decoder network. Following the 45 22 we use AlexNet [26], ResNet18
standard setting in the domain generalization literature 13
20 and ResNet50 20 as backbones for the domain-specific classifiers and the domain-invariant classifier. We use AlexNet for PACS and VLCS, ResNet18 for PACS and Office-Home, and ResNet50 for PACS. AlexNet and ResNet are pre- trained by ImageNet [37 for all the experiments. We use the standard cross-entropy loss the classification loss Lc. For the uncertainty loss Lu. choose the entropy loss. For the reconstruction loss LRs we utilize the pixel-wise (2 loss. A detailed analysis of the loss functions is available in the supplementary material.
Training setting. The encoder-decoder network, the domain-spectin classifiers, and the domain- invariant classifier are all optimized with Stochastic Gradient Descent. The source datasets are split into a training set and a validation set. The learning rate decided by the validation set. We set A = for all the experiments. We give equal weight to the classification loss and the uncertainty loss for training the coomain-specifee classifiers. For X2 and A3, we follow the literature [13 and directly use the
cross validation select their values.
Methods for comparison. We compare our framework with previous domain generalization works including domain invariant based methods
35 g and other state-of siiie--ffth-aaate the-art including data augmentation based methods [34 7 etc. The baseline is defined as the method of empirical risk minimization (ERM). trains a classifier by minimizing the classification loss on all source domains.
(3
45 14
BI
methods [5 4
34
22
meta-learning based methods 4 28 13
Table I: Comparison with existing methods on PACS.
AlexNet
Target
Baseline
CDDGG
30
JiGen
9
Epi-FCR
28
MASF
13
FAR
[4]
DMG
11
455
ER
MetaVIB
[4
RSC
22
LRDG
A
61.13
62.70
67.63
64.70
70.35
68.70
64.65
71.34
71.94
71.62
72.01
C
68.77
71.71
72.30
72.46
70.50
70.29
73.17
75.11
72.97
P
87.96
78.65
89.00
86.10
90.68
90.40
87.31
89.92
91.93
90.88
89.50
58.63
64.45
65.18
65.00
67.33
64.60
71.42
71.15
65.94
66.62
74.86
69.12
68.88
73.38
72.00
75.21
73.54
73.32
75.67
75.74
76.05
77.33
ResNer18
Baseline
Epi-FCR
MASF
DMG
MixStyle
SagNer 34
Stylized 21
StableNer
RSC
LRDG
Target
28
13
111
43
44
22
(ours)
81.74
C
74,24
77.00
77.17
80.38
76.40
78.80
77.66
77.97
79.91
80.31
80.20
P
95.89
93.90
94.99
93.35
96.65
96.
95.47
94.95
96.53
95.99
95.21
73.00
71.69
75.21
71.77
75.90
76.30
81.61
80.50
80.85
84.65
Avg.
79.54
81.50
81.03
81.46
81.38
83.70
83.25
84.32
84.69
85.15
85.48
ResNer50
Baseline
Metareg
DSON
DMG
ER
RSC
MatchDG
SWAD
Fishr
mDSDI
LRDG
Target
358
[
(ours)
A
82.89
87.20
87.04
82.57
87.51
87.89
85.61
89.30
$8.40
$7.70
86.57
C
80.49
79.20
80.62
78.11
79.31
82.16
82.12
83.40
78.70
80.40
85.78
95.99
94.49
S
72.29
70.30
82,90
78.32
76.30
83.35
78.76
82.50
77.80
78.40
86.59
Avg.
86.64
83.37
85.50
86.20
3.2 Main results
PACS contains four domains of Art painting, Cartoon, Photo, and Sketch. These datasets have large domain gaps. The classification results of the previous methods and our framework are shown in Averagely, our framework consistently achieves the best performance in all three backbones
Table I

---

compared with previous works. Especially Sketch, the accuracy of our framework is averagely 3% better than the previous SOTA methods, showing superior performance. Our framework also obtains the best performance on Art painting in AlexNet and maintains the highest accuracy on Cartoon in ResNet50 (ours: 85.78 78% vs. SOTA: 83.40%). This indicates that removing the domain-specific features from the input images is an effective approach for domain generalization. We can also study whether the domain-specific features would benefit or hurt the performance on the unseen target domain by comparing with mDSDI (8) as mDSDI uses the domain-specific features in addition to the domain-invarian features for domain generalization. We can see that our method significantly outperforms mDSDI Cartoon and Sketch, and achieves higher average classification performance than mDSDI in ResNet50. Meanwhile, mDSDI obtains better classification results than ours on Art and Photo. This shows that although Art and Photo may contain similar -eeuaiii-peeiiic features and these features would benefit each other, these domain-specific features would not benefit or even hurt Cartoon and Sketch.
VLCS also contains four domains. Thole the classification accuracy of the domain general- ization methods using the AlexNet backbone. It can be seen that our framework obtains comparable performance to the best-performing methods, and outperforms the prior approaches on LabelMe and Sun. For Office-Home, 8 is used as the backbone. The classification performance is shown in 3 Our framework outperforms the previous methods and achieves the best average performance. Besides, our framework obtains the best performance Art. These experimental results demonstrate that removing the Jomain-specifee features can significantly improve the generalization performance.
Table 2: Comparison with existing methods VLCS using AlexNet backbone.
Bascline
CDDG
JiGen
Epi-FCR
MASF
ER
MetaVIB
Stylized
RSC
LRDG
Target
30
28
133
45]
[4]
X
22
(ours)
61.81
63.06
60.90
64.30
64.90
$8.26
62.66
60.77
61.86
65.53
96.17
94,10
94.78
96.2
96.86
97.61
96.85
63.78
62.10
64.30
65.90
67.64
69.10
67.85
63.42
68.32
69.27
Avg.
72.01
69.59
73.19
72.90
74.11
74.38
74.54
72.31
75.43
75.15
Table 3: Comparison with existing methods Office-Homee using ResNet18 backbones.
Baseline
D-SAMs
JiGen
DSON
MixStyle
SagNet
Stylized
RSC
LRDG
Target
1.5
38
466
22
(purs)
C
46.20
44.37
47.51
45.70
53.40
45.38
52.33
47.90
52.43
71.63
R 73.07
71.45
72.79
74.68
75.90
73.38
75.00
74.54
75.89
Avg.
60.41
60.77
61.20
62.90
65.50
62.34
64.75
63.12
65.75
3.3
Domain divergence
In this section, we investigate the distribution difference among the source domains and the target domain to demonstrate that our framework can effectively reduce domain divergence.
3.3.1 Source domain divergence
To investigate the distribution difference among the source domains, we compute the H-divergence. Following the works of [6] [5), we can approximate the H-divergence by learning algorithm to discriminate between pairwise source domains. For example, with source domains D and D), label the samples of D. by 1, and the samples of D) by 0. We then train a classifier (e.g. linear SVM) to discriminate between these two domains. Given test error of this classifier, Proxy A-distance (PAD) defined as 2(1 2e), which can approximate the H-divergence.
to compute the PAD. For pair of source domains, we combine these domains and construct a new dataset. This dataset randomly split into two subsets of equal size. One subset is used for training and the other one is used for test. We train a collection of linear SVMs (with different values of regularization parameters) on the training set and compute the errors E of all the SVMs the test dataset. The lowest error is used to compute the PAD.
We follow the method from 19 12

---

Proxy A A-distance LRDG
Proxy A-distance on LRDG
2.0
A: Art
2.0
A: Art
(A.C.P).S
1.9
C: Cartoon
1.9
Cartoon
P: Photo
A.C
1.8
P. Photo
18-
S: Sketch
S. Sketch
1.7
1.7
(A.C.S)
[APS)]
1.6
A,P
1.5
1.5
1.4
(C.PS)AA
A.P
1.3
1.5
1.6
1.7
1.8
1.9 2.0
1.3
1.4
1.5
1.6
1.7
1.8
1.9 2.0
Proxy A-distance on baseline
Proxy A-distance on bascline
(a)
(b)
Figure 2: Proxy A-distance (PAD) on PACS, X axis: PAD computed upon the baseline model: y axis: PAD computed upon framework. (a) PAD of pairwise source domains. Blue dots: PAD for Art, Cartoon, and Photo (Sketch as target domain). Orange dots: PAD for Cartoon, Photo, and Sketch (Art target domain). Red dots: PAD for Art, Photo, and Sketch (Cartoon as target domain). Green dots: PAD for Art, Cartoon, and Sketch (Photo as target domain). (b) PAD of pairwise source-target domains. For a target domain (e.g. Art), the corresponding source domain is the closest mixture source domain (e.g. mixture of Cartoon, Photo, and Sketch) to the target domain. The PAD is computed the mixture source domain and the target domain.
Fig. 2a compares the PAD of the raw source domains and the mapped source domains. The experiments are conducted on PACS with the AlexNet backbone. For the raw source domains, we extract features from the baseline model (i.e. the last pooling layer of AlexNet) to train the linear SVMs, while for the mapped source domains, we use features from our domain- -nnvariat classifier. In the figure, each dot represents a pair of source domains (e.g. Art and Photo). It has two values: the PAD of the source domain pair obtained upon the baseline model (x axis) and the PAD of the same pair computed upon our framework (y axis). All the dots are below the diagonal meaning that the PAD values the mapped pairwise source domains are lower than the raw pairwise source domains. With our framework, the mapped source domains become harder to be distinguished, indicating that removing the domain-specific features reduces the distribution difference among the source domains. This also proves that in the generalization risk bound (Eq. [T] would be reduced by framework.
3.3.2 Source- Target domain divergence
We also investigate the distribution difference between the source domains and the target domain. Specifically, we measure the domain divergence between the target domain and the closest mixture source domain D, to the target domain. To obtain this mixture source domain, as defined in Sec. 2.3 we need to find Ti for each source domain D', so that D, = Σ T:D;, where 0 < T, S and ΣN i= N. = 1. Because म; can be any real value in the interval of [0, 1], traversing all values to find the desired n. is impossible. Therefore, We limit the values of т. to the set 0.9, (11 values in total), and find the setting of J that can obtain D.. We traverse all possible settings of [ and obtain all possible mixture source do-
of (0, 0.1, 0.2..............
Table The settings of n. for the closest mixture source domains to the target domains on PACS.
mains D = Ein DiDi For each setting of
(mT), we random sample TiH samples from each source domain Dí and concate- nate all these samples into a mixture source dataset. n, is the number of samples the target domain. By this design, each mixture source domain has an equal number of sam-
Source
Target
D.:
Dł: #2
D πa
De
Cartoon: 0.2
Photo: 0.8
Sketch: O
Art
Art: 0.5
Photo: 0.3
Sketch: 0.2
Cartoon
Art: 0.7
Cartoon: 0.3
Sketch:
Photo
ples to the target domain. Similar to Sec. 3.3.1 we also train classifiers (i.e. linear
Art: 0.1
Cartoon: 0.6
Photo: 0.3
Sketch
SVMs) to discriminate between each mix-
ture source domain and the corresponding target domain. The linear SVMs are trained on image features extracted from the bascline model. We then use the test error to compute the PAD between
8

---

each mixture source dataset and the target dataset. The mixture source domain with the lowest PAD the closest mixture source domain D, to the target domain. The detailed settings of n; for the closest mixture source domains to the corresponding target domains on PACS are listed in Table 4 For convenience, we denote the closest mixture source domain Dr and the target domain Dj together as source-target domain pair.
Fig. 2b shows the PAD of the raw source-target domains and the mapped source-target domains. Similar to Sec. 3.3.T] for the raw source-target domains, we extract the image features from the baseline model to train the linear SVMs. To compute the PAD of the mapped cauree-target domains, we extracted the image features from our domain invarrinn classifier to train the linear SVMs. In the figure, each dot represents source-targe domain pair (e.g. (Cartoon, Photo, Sketch), Art). We can see that all the dots are below the diagonal. The PAD values of the mapped source- target pairs are lower than the raw source-target pairs. This indicates that the distribution difference between the source domains and the target domain is reduced by our framework. 7y in the generalization risk bound (Eq. [TI would be lowered. Removing the domain- -peeif- features from the source domains can also reduce the distribution difference between the source domains and the target domain.
In summary, our framework can reduce the distribution difference not only among the source domains but also between the source domains and the target domain. This also demonstrates that our framework could effectively lower the value of the generalization risk bound by reducing e and %.
4 Related work
Domain generalization is a challenging task that requires models to be well performed on unseen domains. One common approach is to learn domain-invariant features among the source domains. Previous methods aim to distill the DNNs that the domain-specific features shall be effectively removed. Muandet et al. [33] propose to reduce the domain dissimilarity by kernel-based method. Ghifary et al. [8]] reduce dataset bias by extracting features that are shared among the source domains with a multi-task autoencoder network. Li et al. (29 utilize Maximum Mean Discrepancy (MMD) on adversarial autoencoders to align the distributions across source domains. Li et al. [30] design an end-to-end conditional invariant deep neural network that minimizes the discrepancy of conditional distributions across domains. Arjovsky et al. (3] develop Invariant Risk Minimization (IRM) that uses a causal mechanism to obtain the optimal invariant classifier upon the representation space. Chattopadhyay et al. [11] propose to learn -opeii- binary masks to balance the domain-invariant and domain-specific features for the prediction of unseen target domains. Zhao et al. 45 propose entropy regularization method to learn the domain-invariant conditional distributions by using a classification loss and a domain adversarial loss. Du et al. (14) develop probabilistic meta-learning method that learns domain-invann innarinnnvvaiant representations with meta variational information bottleneck principle derived from variational bounds of mutual information. Mahajan et al. (3I] assume that domains are generated by mixing causal and non-causal features and that the same object from different domains should have similar representations. Based on this, they propose a new method called MatchDG build a domain- invariant classifier by matching similar inputs. Rame et al. [35 match the gradients among the source domains to minimize domain invariance. Unlike the above works, Bui al. [ assume that, besides the domain-invariane features, some domain- information for the target domain. However, this cannot always be guaranteed since the target domain is unseen. For example, the backgrounds in the domain Photo may benefit the domain Art, but they would not benefit or even hurt the domain Sketch. Our framework follows the common assumption that the domain-invariant features are generalized across domains, regardless of the effect of the fomain-specifis features [30, 45)
features, but they do not clearly inform the
features also provide useful
Recent papers demonstrate that CNNs tend to classify objects based on features from superficial local textures and backgrounds, while humans rely on global object shapes for classification 23 17 To address this issue, some methods aim to capture the global object shapes from the images. These methods are proposed based on the assumption that the local textures and backgrounds are the domain- specific features, and the global object shapes are the domain-invariant features. Wang et al. [42 extract semantic representations by penalizing features extracted with gray-level co-occurrence matrix (GLCC) which are sensitive to texture. Wang et al. [41] penalize the earlier layers of CNNs from learning local representations and make the CNNs rely on the global representations for classification. Although addressing the superficial local features is a promising approach, the superficial local

---

features may be one kind of domain- -ppeciffc features and other forms of domain-specific features may also exist. Compared with these methods, our framework proposed address the more general domain-specific features rather than the superficial local features.
5 Conclusion
In this work, we propose a new approach that aims to explicitly remove domain-specific features for domain generalization. To this end, we develop a novel domain generalization framework that learns the domain-invariam features by actively removing the domain-specific features from the input images. We also experimentally verify the reduced domain divergence among the source domains and the target domain brought by our approach. Experiments show that our framework achieves strong performance on various datasets compared with existing domain generalization methods.
Despite the advantages of our framework, it has some potential limitations to be further addressed. We need train the same number of domain-specific classifiers the source domains. When there are more source domains, more computational resources will be required to train the domain-specific classifiers. This may be addressed by designing novel domain-specifie classifier that can learn the domain-specific features of multiple source domains simultancously. Another limitation of our framework is that it cannot remove the domain-specific features of the unseen target domain. These domain-specific features should also be removed since they would negatively affect the classification performance. For example, our framework performs slightly worse than the baseline when Photo is the target domain (as shown in [Tble(]) This may be because Photo contains rich domain -peccff features compared with the source domains, and our framework would make incorrect predictions due to these domain-specific features. Besides, this result also shows that domain-specific knowledge useful for Photo. As the target domain is not available during training, how to remove the domain- specific features from the target domain and whether to retain the domain-specifi features of the source domains will be challenging issues to addressed. One possible future work may be to remove the domain-specific features in a latent feature space. To achieve this, the framework may need to be adjusted, including the domain- encoder-decoder network incurs extra computational overhead, but performing latent space may have the benefit that we may no longer need the mncoder-decoder network and the overall framework can be computationally more efficient.
classifiers and the domain-invariant classifier. The
Acknowledgment
Yu Ding was supported by CSIRO Data61 PhD Scholarship and the University of Wollongong Inter- national Postgraduate Tuition Award. This research was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI) and the CSIRO Accelerator
References
[1] Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, and Mario Marchand. Domain-adversarial neural networks. arXiv preprint arXiv:1412.4444 2014.
[2] Isabela Albuquerque, João Monteiro, Mohammad Darvishi, Tiago Falk, and Ioannis Mitliagkas. Generalizing to unseen domains via distribution matching. arXiv preprint 1:Xiv:991100000/ 2020.
[3] Martin Arjovsky, Léónn Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk mini- mization. arXiv preprint arXiv:197.002893 2019.
[4] Yogesh Balaji, Swami Sankaranarayanan, and Rama Chellappa. Metareg: Towards domain generalization using meta-regularization.. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, pages 1006-1016, 2018.
[5] Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jen- nifer Wortman Vaughan. theory of learning from different domains. Machine learning, 19(1):151-175, 2010.
10

---

[6] Shai Ben-David, John Blitzer, Koby Crammer, Fernando Pereira, et al. Analysis of represen- tations for domain adaptation. Advances in neural information processing systems, 19:137, 2007.
[7] Francesco Cappio Borlino, Antonio D D'Innocente, and Tatiana Tommasi. Rethinking domain generalization baselines. In 25th International Conference on Pattern Recognition, pages 9227-9233. IEEE, 2021.
[8] Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Exploiting domain-specific features to enhance domain generalization. Advances in Neural Information Processing Systems, 34:21189- 21201, 2021.
[9] Fabio M Carlucci, Antonio D' D'Innocente, Silvia Bucci, Barbara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2229-2238, 2019.
[10] Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheo Cho, Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34, 2021.
[11] Prithvijit Chattopadhyay, Yogesh Balaji, and Judy Hoffman. Learning to balance specificity and invariance for in and out of domain generalization. In European Conference on Computer Vision, pages 301-318. Springer, 2020.
[12] Minmin Chen, Zhixiang Xu, Kilian Weinberger, and Fei Sha. Marginalized denoising autoen- coders for domain adaptation. In International Conference Machine Learning, 2012.
[13] Qi Dou, Daniel Coelho de Castro, Konstantinos Kamnitsas, and Ben Glocker. Domain general- ization via model-agnostic learning of semantic features, In Advances in Neural Information Processing Systems, pages 6450-6461, 2019.
[14] Yingjun Du, Jun Xu, Huan Xiong, Qiang Qiu, Xiantong Zhen, Cees GM Snock, and Ling Shao. Learning to learn with variational information bottleneck for domain generalization. In European Conference on Computer Vision, pages 200-216. Springer, 2020.
[15] Antonio D' D'Innocente and Barbara Caputo. Domain generalization with domain-specific ag- gregation modules. In German Conference on Pattern Recognition, pages 187-198. Springer, 2018.
[16] Robert Geirhos, Jörn- Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning deep neural networks. Nature Machine Intelligence, 2(1 11:61)6666556 6635, 2020.
[17] Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix Wichmann, and Wieland Brendel. Imagenet- trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. In International Conference on Learning Representations, 2018.
[18) Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, and David Balduzzi. Domain gen- eralization for object recognition with multi-task autoencoders. Proceedings of the IEEE international conference computer vision, pages 2551-2559, 2015.
[19] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Domain adaptation for large-scale sentiment classification: A deep learning approach. In International Conference on Machine Learning, 2011.
[20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[21] Katherine Hermann, Ting Chen, and Simon Kornblith. The origins and prevalence of texture bias convolutional neural networks. Advances in Neural Information Processing Systems, 33, 2020.
11

---

[22] Zeyi Huang, Haohan Wang, Eric P. Xing, and Dong Huang. Self-challenging improves cross- domain generalization. In European Conference on Computer Vision, 2020.
[23] Jason Jo and Yoshua Bengio. Measuring the tendency of cnns to learn surface statistical regularities. arXiv preprint (rXiv:/711.115000 2017.
[24] Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pages 694-711. Springer, 2016.
[25] Daniel Kifer, Shai Ben-David, and Johannes Gehrke. Detecting change in data streams. In Proceedings of the Thirtieth international conference on Very large data Vores-Voluue 30, pages 180-191, 2004.
[26] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84-99) 90, 2017.
[27] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision, pages 5542-5550, 2017.
[28] Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, and Timothy M Hospedales. Episodic training for domain generalization. In Proceedings of the IEEE International Confer- ence on Computer Vision, pages 1446-1455, 2019.
[29] Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, pages 5400-5409, 2018
[30] Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao. Deep domain generalization via conditional invariant adversarial networks. In Proceedings of the European Conference on Computer Vision, pages
2018.
[31] Divyat Mahajan, Shruti Tople, and Amit Sharma. Domain generalization using causal matching. In International Conference on Machine Learning, pages 7313-7324. PMLR, 2021.
[32] Matthias Minderer, Olivier Bachem, Neil Houlsby, and Michael Tschannen. Automatic shortcut removal for self-supervised representation learning. International Conference on Machine Learning, 2020.
[33] Krikamol Muandet, David Balduzzi. and Bernhard Schölkopf. Domain generalization via invariant feature representation. In International Conference on Machine Learning, pages 10-18. PMLR, 2013.
[34] Hyeonseob Nam, HyunJae Lee, Jongehan Park, Wonjun Yoon, and Donggeun Yoo. Reducing domain gap by reducing style bias. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8690-8699, 2021.
[35] Alexandre Rame, Corentin Dancette, and Matthieu Cord. Fishr: Invariant gradient variances for out-of-distribution generalization. International Conference on Machine Learning, pages 18347-18377. PMLR, 2022.
[36] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net:
Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and cosiuter-assisted intervention, pages 234-241. Springer, 2015.
[37] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, al. Imagenet large scale visual recognition challenge. International journal of computer vision, 1 115(3)2111-555 2015.
[38] Seonguk Seo, Yumin Suh, Dongwan Kim, Geeho Kim, Jongwoo Han, and Bohyung Han. Learning to optimize domain specific normalization for domain generalization. European Conference Computer Vision, pages 68-83. Springer, 2020.
12

---

[39] Antonio Torralba and Alexei A Efros. Unbiased look at dataset bias. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition, pages 1521-1528, 2011.
[40]
Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018-5027, 2017.
[41] Haohan Wang, Songwei Ge. Zachary Lipton, and Eric Xing. Learning robust global represen- tations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pages 10506-10513 2019.
[42] Haohan Wang, Zexue He, and Eric Xing. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations, 2019.
[43] Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, and Mare Niethammer. Robust and generaliz- able visual representation learning via random convolutions. In International Conference on Learning Representations, 2020.
[44] Xingxuan Zhang, Peng Cui, Renzhe Xu, Linjun Zhou, Yue He, and Zheyan Shen. Deep stable learning for out-of-distribution generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5372-5382, 2021.
[45] Shanshan Zhao, Mingming Gong, Tongliang Liu, Huan Fu, and Dacheng Tao. Domain gener- alization via entropy regularization. Advances in Neural Information Processing Systems, 33, 2020.
[46] Kaiyang Zhou, Yongxin Yang. Yu Qiao, and Tao Xiang. Domain generalization with mixstyle. In International Conference on Learning Representations, 2021.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] In the Appendix.
(c) Did you report error bars (e.g., with respect the random seed after running experi- ments multiple times)? [Yes] In the Appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] In the Appendix.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] In the Appendix.
(b) Did you mention the license of the assets? [Yes] In the Appendix.
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [Yes] In the Appendix.
13

---

(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsoureing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "ionaUR66Tj", "reviewer": "Reviewer_8K1L", "summary": "The manuscript describes a new framework for training models that are general for multiple domains (domain generalisation). Their protocol is to (1) train one classifier for each domain, which are made domain-specific by enforcing a random guess for domains that they are not specialised to, (2) train an auto encoder where the decoding process is constrained to generate images that “confuse” all of the domain specific classifiers and at the same time (3) train a new classifier on top of those domain-agnostic images.", "questions": "Why does Figure 2 not include PAD of pairwise domains with Photo and Cartoon as target domains? \n \nDid the authors consider performing a similar method on a latent space instead of directly on the image space? (This question or its answer do not interfere with my rating)\n", "limitations": "The limitations of the model are addressed in the Conclusion. The authors discuss the need for training separate classifiers for each source domain and that the model is not able to remove domain-specific features that regard unseen domains. I would add the limitation that some domain-specific knowledge is useful for some domains, as we can see that this model does not outperform others on more complex domains such as Photo and sometimes Art.\n\nEDIT (post-discussion): I will also add this here for completeness. Per the discussion below and with other reviewers, the authors have acknowledged that the removal of domain-specific features is conditional on these features being learned by the domain-specific classifiers via the optimisation in Eq. (8). This is expected to work given the design of Eq. (8) but it is not *guaranteed*, and therefore this assumption configures an expected limitation of the work that future readers and users should keep in mind. ", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and easy to follow. The presented idea for generating domain-invariant images and training domain specific classifiers is intuitive and the entire package seems to bring about improvements on average to performance on the baselines datasets. The authors have made comparisons against state-of-the-art studies and compared across different backbones.\n\nThe use the generalisation bound, and of the Proxy A-distance (PAD) is very welcome here for showing that the model truly generalises. However, the application used by the authors employ the features of an AlexNet network for the unmapped domains and the features of their domain-invariant classifier for the mapped domains; this makes it hard to assess whether their mapping was effective in reducing the H-divergence between domains. It would probably be more adequate to use the same features for both to allow for a fairer comparison. I understand this is not a simple choice given the features of an AlexNet will be more discriminative for the photo domain (and this will probably also apply for other pre-trained networks) and the features of the domain-invariant classifier would be less discriminative for unmapped domains since it was trained for mapped versions. \n\nEven though it is the intuitive conclusion that the framework would remove domain-specific features, it is not guaranteed to do so. Different classifiers could work with different manifolds of the original image space to do their classification and the uncertainty loss could force each classifier to use their own “space” to perform classification, ignoring or not domain-specific features from other domains; this is similar to how SVMs with different kernels could solve the same classification tasks and even use spaces with the same dimensionality. This non-guarantee is important to state since the authors state that they reduce \\lambda (lines 174-176) and “prove” that they reduce e in the generalisation risk bound; maybe those statements could be toned down with “probably” or “indicates that … reduces”. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "A9OylQZF4Yx", "reviewer": "Reviewer_MBUN", "summary": "This paper introduces a novel framework, namely learning and removing domain-specific features for generalization (LRDG), allowing the trained model can extract only domain-invariant features in order to improve the out-of-domain generalization performance. In particular, in LRDG, for each source domain, the training process of the domain-specific classifier is based on the classification loss and the uncertainty loss on other source domains. The experimental results show that the proposed method can produce better classification accuracy compared to several existing methods.", "questions": "1. A mathematical definition for domain-specific features?\n2. A theoretical guarantee capability to learn the real domain-specificity of the proposed LRDG?\n3. A comparison with the method in [3], which indicates the usefulness of domain-specific features in domain generalization?", "limitations": "N/A", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: The proposed framework for learning domain-specific features based on an autoencoder and uncertainty loss in the paper is novel in domain generalization. \n\nQuality: The technical contributions of the paper are relatively insignificant due to the lack of a mathematical definition for domain-specific features and a theoretical guarantee for the capability to learn the real domain-specificity of the proposed LRDG. Although I appreciate the theoretical result about the generalization bound in Theorem 2, it does not support the main claim of the paper.\n\nClarity: The paper is quite well-written and easy to follow. \n\nSignificance: My major concern is about the real benefit of the proposed method. In particular, compared to the direct domain-invariant learning approaches (see e.g., [1,2]) the LRDG framework seems to be more complicated and more computationally expensive since requires the training of an autoencoder.  Moreover, the main idea of LRDG is to learn and eliminate the domain-specific features, while some recently published papers (e.g., [3]) show that effectively combining domain-specific and domain-invariant features are also useful for domain generalization. \n\n[1] @inproceedings{li2018deep, title={Deep domain generalization via conditional invariant adversarial networks}, author={Li, Ya and Tian, Xinmei and Gong, Mingming and Liu, Yajing and Liu, Tongliang and Zhang, Kun and Tao, Dacheng}, booktitle={Proceedings of the European Conference on Computer Vision (ECCV)}, pages={624--639}, year={2018} } \n\n[2] @inproceedings{hu2020domain, title={Domain generalization via multidomain discriminant analysis}, author={Hu, Shoubo and Zhang, Kun and Chen, Zhitang and Chan, Laiwan}, booktitle={Uncertainty in Artificial Intelligence}, pages={292--302}, year={2020}, organization={PMLR} }\n\n[3] @article{bui2021exploiting,\n  title={Exploiting domain-specific features to enhance domain generalization},\n  author={Bui, Manh-Ha and Tran, Toan and Tran, Anh and Phung, Dinh},\n  journal={Advances in Neural Information Processing Systems},\n  volume={34},\n  pages={21189--21201},\n  year={2021}\n}", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0_5hZNVIWae", "reviewer": "Reviewer_T9VV", "summary": "The paper proposes a domain generalization method. The key idea is to remove domain-specific features by first training domain-specific classifiers for all domains then training encoder-decoder network that transforms an input image to a domain-invariant version based on them. Specifically, the encoder-decoder network is trained together with another domain-invariant classifier so that the domain-specific classifiers cannot discriminate the classes of transformed images but the domain-invariant classifier can classify them. Experiments show that the model improves the generalization performance from the baselines on PACS, VLCS, and Office-Home with several different backbone architectures.", "questions": "- It would be interesting to visualize output of the encoder-decoder network. It will provide more insights how the method behaves.\n- How does the encoder-decoder architecture affect the performance?\n- Which data augmentation was used for training? Was the same augmentation used for all the comparing methods?", "limitations": "The authors did not address the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "1. Strengths\n\n- The idea to learn domain-invariant features has been studied in several existing work. Among them, the paper looks similar to the Epi-FCR [28], which also uses domain-specific classifiers to train a domain-invariant feature extractor. This paper explicitly designs a training scheme to remove domain-specific features and the effect looks pretty clear.\n- The proposed method consistently improves the baseline on PACS, VLCS, and Office-Home with different backbone architectures.\n- The paper is clearly written and easy to follow.\n\n2. Weaknesses\n- I think maximizing the classifier uncertainty of a domain-specific classifier does not necessarily guarantee that the domain-specific features are removed. For example, assume an input image has two channels and the trained domain-specific classifier F_1 uses only the first channel. Since the classifier is fixed, the encoder-decoder network can cheat to include domain-specific features in the second channel, so that it can perform well on the classifier F, which was originally intended to be domain-invariant. At the same time, encoder-decoder network can be trained to output the first channel constant so that the classifier F_1 cannot discriminate classes properly.\n- Minor weakness is that the method needs to pass the input image through the encoder-decoder network and it makes an overhead in inference time.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept