Domain Generalization by Learning and Removing Domain-specific Features
Anonymous Author(s)
AffiliationAddress
Abstract
Deep Neural Networks (DNNs) suffer from domain shift when the test datasetfollows a distribution different from the training dataset.
Domain generalizationaims to tackle this issue through learning a model that can generalize to unseendomains.
In this paper, we propose a new approach that aims to explicitly remove domain-specific features for domain generalization.
Following this approach,we propose a novel framework called Learning and Removing Domain-specificfeatures for Generalization (LRDG) that learns a domain-invariant model by tactically removing domain-specific features from the input images.
Specifically, wedesign a classifier to effectively learn domain-specific features for each sourcedomain, respectively.
We then develop an encoder-decoder network to map eachinput image into a new image space where the learned domain-specific featuresare removed.
With the images output by the encoder-decoder network, anotherclassifier is designed to learn the domain-invariant features to conduct image classification.
Extensive experiments demonstrate that our framework achieves superiorperformance compared with the state-of-the-art methods.
Introduction
Deep Neural Networks (DNNs) have achieved great performance in computer vision tasks [26].
However, the performance would drop if the test dataset follows a distribution different from thetraining dataset.
This issue is also known as domain shift [42].
Recent research has found that the
DNNs tend to learn decision rules differently from humans [16, 21, 15].
For example, in ImageNetbased [37] image classification tasks, the Convolutional Neural Networks (CNNs) tend to learnlocal textures to discriminate objects, while we humans could use the knowledge of global objectshapes as cues.
The features learned by the DNNs may only belong to specific domains and are notgeneralized for other domains.
For example, in real-world photos, objects belonging to the samecategory have similar textures, but in sketches [27], objects are only drawn by lines and containno texture information.
For a CNN that uses textures to discriminate objects in the photos, poorperformance can be expected when it is applied to the sketches.
This situation calls for the DNNsthat can learn features invariant across domains instead of learning features that are domain-specific.
In this paper, we focus on the research topic of domain generalization and follow the multiple sourcedomain generalization setting in the literature.
Its goal is to train a model that can perform well onunseen domains.
In this setting, we can access multiple labeled source domains and one or moreunlabeled target domains.
All the source and target domains share the same label space.
Duringthe training process, the source domains are available but the target domains are unseen.
The targetdomains are only provided at the test phase.
One typical approach to domain generalization is to learn domain-invariant representations acrossdomains [18, 30, 45, 3, 10, 13, 48, 31, 35].
This approach is based on the assumption that each domain has its domain-specific features and all domains share domain-invariant features.
For example,textures are domain-specific features for the photos but shapes are domain-invariant features forboth photos and sketches.
Previous works propose methods that seek to distill domain-invariantfeatures.
Although demonstrating promising performance, these methods do not clearly inform thedeep networks that domain-specific features shall be effectively removed.
Instead, it is only hoped thatthey would be removed through achieving the final goal of learning domain-invariant features.
Thelack of this clear guidance to the network may affect its learning efficacy.
In this paper, we proposea new approach that aims to explicitly remove domain-specific features in order to achieve domaingeneralization.
As indicated above, the CNNs tend to learn domain-specific features rather thandomain-invariant features for classification.
To prevent this from taking place, we actively removedomain-specific features and guide the CNNs to learn domain-invariant features for classification.
Following this approach, we propose a novel framework: Learning and Removing Domain-specificfeatures for Generalization (LRDG).
Our framework consists of domain-specific classifiers, an encoder-decoder network, and a domaininvariant classifier.
Each domain-specific classifier can effectively learn domain-specific featuresfrom one source domain.
Specifically, a domain-specific classifier is designed to discriminate theimages across different classes within one particular source domain.
At the same time, this classifieris required to be unable to discriminate the images across different classes within any other sourcedomain.
Each source domain therefore corresponds to one domain-specific classifier under thisdesign.
The encoder-decoder network in our framework maps the input images into a new imagespace where the domain-specific features are to be removed from the input images by utilizing thelearned domain-specific classifiers.
The transformed images are expected to contain much fewerdomain-specific features compared with the original input images.
The domain-invariant classifieris then appended to the encoder-decoder network and trained with the transformed images.
Bythis design, the encoder-decoder network actively removes the domain-specific features and thedomain-invariant classifier will be better guided to learn the domain-invariant features.
Once trained,the encoder-decoder network and the domain-invariant classifier will be used for the classification ofunseen target domains.
It is worth noting that our framework is different from the data augmentation based methods fordomain generalization [46, 34, 49, 7].
Our framework aims to remove domain-specific featuresfrom the input images while data augmentation based methods generate various images with noveldomain-specific features.
Besides, our framework just transforms input images into a new imagespace and does not augment them to enlarge the training dataset.
We demonstrate the effectiveness of our framework with experiments on three benchmarks indomain generalization.
Our framework consistently achieves state-of-the-art performance.
We alsoexperimentally verify that our framework effectively reduces the distribution difference among thesource and target domains according to the generalization risk bound in the literature [2].
Proposed framework
Assuming that we are given N source domains D s
= { D 1 s , D 2 s , . . .
, D Ns } with each domain following 75 different distributions.
For each domain (dataset), D is = { ( x ij , y ij ) } n i j =1 where n i is the number ofsamples in domain D is , and ( x ij , y ij ) is the data-label pair for the j th sample in the i th domain.
Following the literature, we assume that all source and target domains share the same label space.
The goal of domain generalization is to use these source domains D s to learn a model for the unseentarget domain D t .
Our work is inspired by recent work [32], where it uses a "lens" network (i.e. image-to-imagetranslation network) to remove "shortcuts" (low-level visual features that a CNN can quickly learn,such as watermarks and color aberrations) from input images in a self-supervised learning task.
Differently, our work focuses on removing domain-specific features from the input images for thedomain generalization task.
We use an encoder-decoder network similar to the "lens" network, butwe design a different method to leverage the encoder-decoder network to remove the domain-specificfeatures.
In this section, we illustrate our framework in detail.
We also provide theoretical analysisfor our framework.
Fig.
1 gives an overview of the entire framework.
𝐹𝐿 𝐶 𝐿 𝑈 : Uncertainty Loss 𝐿 𝐶 : Classification Loss
M 𝐱𝐱𝐱 N freeze 𝐱𝐱𝐱 𝑁 … …
𝐿 𝑈 𝐿 𝑈 𝐹𝐿 𝑈 freeze 𝐱𝐱𝐱 𝑁 … …
𝐿 𝐶 𝐿 𝑈 ∙∙∙ … … 𝐹𝐹𝐹 𝑁 𝐹 𝐿 𝑈 𝐿 𝑈 𝐿 𝑈 𝐿 𝐶 𝐳𝐳𝐳 N 𝐿 𝑅
M 𝐱 t 𝐹 prediction 𝐱 𝑖 : Images from source domain 𝑖 𝐹 𝑖 : Domain-specific Classifier for source domain 𝑖
M : Encoder-decoder Network
F : Domain-invariant Classifier 𝐳 𝑖 : Transformed Images of 𝐱 𝑖 𝐿 𝑅 : Reconstruction Loss 𝐱 𝑡 : Images from target domain
[Figure]
2.1 Learning domain-specific features
Our framework starts by training N individual domain-specific classifiers F S
= { F 1 , F 2 , . .
, F N } 90 in which the i th classifier F i is designed to only use the domain-specific features from the i th sourcedomain D is to discriminate images.
The domain-specific classifiers F S should not use domaininvariant features as cues.
In other words, F i is expected to be able to effectively discriminateimages within D is
but it should be difficult for F i to discriminate images within any other domain.
Domains excluding D is are used to maximize the classification uncertainty or adversarially increasethe difficulty of classification for F i .
The classification performance of F i on domains excluding D isshould be similar to a random guess.
Specifically, the classifier F i is trained by minimizing a classification loss L F S C on D is ,
[Equation]
and maximizing an uncertainty loss L F S U on the remaining domains { D 1 s , . .
D i − 1 s
[Equation]
D
[Equation] [Equation]
D
[Equation] [Equation]
[Equation]
where θ i denotes the parameters of the classifier F i .
L C and L U are the classification loss functionand the uncertainty loss function, respectively.
We use cross-entropy loss as the classification loss.
For the uncertainty loss, since we aim to make the prediction similar to a random guess, we useentropy loss:
[Equation]
where C is the number of classes and p ( y = l | F i ( x kj ; θ i )) denotes the probability of x kj belonging toclass l .
Least likely loss [32] is also an alternative of the entropy loss.
The classifier first predicts an image and obtains the probabilities of all the classes.
The class with the lowest probability is calledleast likely class.
This image is assigned with a label of this class.
Then we train the classifier topredict the least likely class.
The least likely loss is
[Equation]
ˆ
[Equation]
However, experiments show that the entropy loss can better achieve the classification randomnessthan the least likely loss, so we use entropy loss as the default uncertainty loss.
After training, we freeze the parameters θ of these classifiers F S and use them to learn domaininvariant features.
2.2 Removing domain-specific features
To remove domain-specific features from the input images, we utilize an encoder-decoder network Mthat maps the images into a new image space Z .
The output images are fed into the domain-specificclassifiers F S and a new domain-invariant classifier F .
Unlike the training of the domain-specific classifier F i where the source domain D is is used forminimizing the classification loss, on the contrary, the source domain D is in this step is used tomaximize the uncertainty loss L MU
[Equation]
The parameters θ i of F i are frozen and the parameters θ M of the encoder-decoder network M aretrained.
Maximizing the uncertainty loss forces the output image
z i = M
( x i ) to contain a minimumamount of domain-specific features.
In doing so, the encoder-decoder network can remove thedomain-specific features in the input images x and retain domain-invariant features in the outputimages z .
To maintain the overall similarity between the input and output images, we add a reconstruction loss
L MR for the encoder-decoder network,
[Equation]
where L R is the reconstruction loss function.
We use pixel-wise l 2 loss as the default reconstructionloss for its simplicity and reasonably good performance.
Other reconstruction losses could also beemployed, such as pixel-wise l 1 loss and perceptual losses [24].
Detailed discussion is available inthe supplementary material.
We then minimize the classification loss L FMC by training the domain-invariant classifier F on allsource domains
[Equation]
where θ F are the parameters of the domain-invariant classifier F .
This classification loss L FMCalso updates the encoder-decoder network to prevent the encoder-decoder network from losing thedomain-invariant features due to the uncertainty loss.
The uncertainty loss also has the potentialto remove domain-invariant features if it is difficult to separate the domain-specific features fromdomain-invariant features.
Overall, when training domain-specific classifiers we optimize
[Equation] [Equation]
and when learning domain-invariant features, we optimize
[Equation]
where λ 1 , λ 2 and λ 3 are hyperparameters that control the relative weight of these losses.
For convenience, we denote the encoder-decoder network M and the domain-invariant classifier F asa domain-invariant model.
At the testing phase, the domain-invariant model is used for classifyingimages of the target domain.
2.3 Explanation of LRDG with respect to existing theory
We first introduce the generalization risk bound for domain generalization [2] and then further explainthe effectiveness of our framework with respect to this.
Theoretically, the corresponding task for a domain is defined as a deterministic true labeling functionf , where f : X → Y .
X and Y are the input space and the label space, respectively.
We denote the 149 space of the candidate hypothesis as
H , where a hypothesis h : X → Y .
The risk of the hypothesis h 150 on a domain
D is defined as:
[Equation]
where L : Y × Y → R + measures the difference between the hypothesis and the true labeling 152 function.
Following [2], for the source domains {D 1 s , D 2 s , . . .
, D Ns } , we define the convex hull Λ S of the sourcedomains as a set of mixture source distributions: Λ S
= { ¯ D : ¯ D ( · ) =
(cid:80) Ni =
1 π i D is ( · ) , 0 ≤ π i ≤1 , (cid:80) Ni
=
π i = 1 } .
We also define ¯ D t ∈ Λ S as the closest domain to the target domain D t .
¯ D t isgiven by arg min π 1 ,...,π N d H [ D t , (cid:80)
Ni =
1 π i D is ] , where d H [ · , · ] is H -divergence [25] that quantifies the 157 distribution difference of two domains.
We use the following generalization risk bound [2] for thetarget domain D t .
Theorem 1 [2].
Given the previous setting, the following inequality holds for the risk R t [ h ] , ∀ h ∈
H 160 for any domain
D t :
[Equation]
where γ = d H [ D t , ¯ D t ] , ϵ = sup
i,j ∈ [ N ] d H [ D is , D js ] and λ π is the minimum sum of the risks achievedby some h ∈ H on D t and ¯ D t .
γ measures the distribution difference between the source domains 163 and the target domain.
ϵ is the maximum pairwise
H -divergence among source domains.
Theorem 1shows that the upper bound for the target depends on γ and ϵ .
We show that our framework can effectively lower the value of this generalization risk bound for agiven domain generalization task.
Recall that our encoder-decoder network maps the input images intoa new image space.
We denote the mapped source domains as { (cid:98) D 1 s , (cid:98) D 2 s , . . .
, (cid:98) D Ns } and the mappedtarget domain as (cid:98) D t .
With the domain-specific classifiers, the domain-specific features are removedfrom the source domains and the mapped source domains tend to have common domain-invariantfeatures.
As a result, the mapped source domains { (cid:98) D 1 s , (cid:98) D 2 s , . .
, (cid:98) D Ns } would have smaller distributiondifference than the raw source domains, i.e. d H [ (cid:98) D is , (cid:98) D js ]
≤ d H [ D is , D js ] .
ϵ in Eq.
11 would probably 172 be reduced.
After removing the domain-specific features for each source domain, the mapped targetdomain (cid:98) D t is closer to the mapped source domains, so our framework would also tend to reduce γin Eq.
Concerning Theorem 1, these changes provide a principled explanation and warrant tothe effectiveness of the proposed framework.
We will demonstrate these changes in the experimentsection (Sec. 3.3).
Experiments
We evaluate our framework on three benchmark datasets and compare the performance with previousmethods.
After that, we study the domain divergence among the source and target domains.
3.1 Datasets and settings
Datasets.
We evaluate our framework on three object recognition datasets for domain generalization.
PACS [27] contains four domains: Photo ( P ), Art Painting ( A ), Cartoon ( C ) and Sketch ( S ) witheach domain covering seven categories including dog, elephant, giraffe, guitar, horse, house, andperson.
VLCS [42] also has four domains: PASCAL VOC 2007 ( V ), LabelMe ( L ), Caltech ( C ) and
Sun ( S ) and images belong to five categories of bird, chair, car, dog, and person.
Office-Home [43]has images from 65 categories over four domains including Art ( A ), Clipart ( C ), Product ( P ), and
Real-World ( R ).
For each dataset, following the literature, the experimental protocol is to considerthree domains as the source domains and the remaining one as the target domain.
Networks and loss functions.
We use U-net [36] for the encoder-decoder network.
Following thestandard setting in the domain generalization literature [12, 48, 22], we use AlexNet [26], ResNet18[20] and ResNet50 [20] as backbones for the domain-specific classifiers and the domain-invariantclassifier.
We use AlexNet for PACS and VLCS, ResNet18 for PACS and Office-Home and ResNet50for PACS.
The AlexNet and ResNet are pre-trained by ImageNet [37] for all experiments.
We use thestandard cross-entropy loss as the classification loss L C .
For the uncertainty loss L U , we choose theentropy loss, and for the reconstruction loss L R , we utilize the pixel-wise l 2 loss.
A detailed analysisof the loss functions is available in the supplementary material.
Training setting.
The encoder-decoder network, the domain-specific classifiers, and the domaininvariant classifier are all trained with Stochastic Gradient Descent.
The source datasets are split intoa training set and a validation set.
The learning rates are decided by the validation set.
We set λ 1 = 1for all the experiments.
We give equal weight to the classification loss and the uncertainty loss fortraining the domain-specific classifiers.
For λ 2 and λ 3 , we follow the literature [12, 4] and directlyuse the leave-one-domain-out cross-validation to select their values.
Methods for comparison.
We compare our framework with previous domain generalization worksincluding domain-invariant based methods [30, 44, 10, 48, 13, 31, 35] and other state-of-the-artmethods [14, 4, 8, 28, 12, 49, 34, 22, 7, 47, 9] including data augmentation based methods [34, 49, 7],meta-learning based methods [4, 28, 12], etc.
The baseline is defined as the method of empirical riskminimization (ERM).
It trains a classifier by minimizing the classification loss on all source domains.
3.2 Main results
PACS contains four domains of Art painting, Cartoon, Photo, and Sketch.
These datasets have largedomain gaps.
The classification results of previous methods and our framework are shown in Table 1.
Averagely, our framework consistently achieves the best performance in all three backbones comparedwith previous works.
Especially on Sketch, the accuracy of our framework is average 3% better thanthe previous SOTA methods, showing superior performance.
Our framework also obtains the bestperformance on Art painting in AlexNet and maintains the highest accuracy on Cartoon in ResNet50(ours: 85 .
78% vs. SOTA: 83 .
This indicates that removing the domain-specific features fromthe input images is an effective method for domain generalization.
[Table] [Table]
ours)
[Table] [Table]
[Table] [Table]
[Table] [Table]
[Table]
VLCS dataset also contains four domains.
Table 2 shows the classification accuracy of the domaingeneralization methods using the AlexNet backbone.
It can be seen that our framework obtainscomparable performance to the best-performing methods, and also outperforms prior approacheson LabelMe and Sun.
For Office-Home, the ResNet18 is used as the backbone.
The classificationperformance is shown in Table 3.
Our framework outperforms previous methods and achieves the best average performance.
Besides, our framework obtains the best performance on Art (more than 1%improvement compared with the SOTA), demonstrating that removing the domain-specific featurescan significantly improve the generalization performance.
[Table] [Table]
3.3 Domain divergence
In this section, we investigate the distribution difference among the source domains and the targetdomain to demonstrate that our framework can effectively reduce domain divergence.
3.3.1 Source domain divergence
To investigate the distribution difference among the source domains, we compute the H -divergence.
Following the works of [6, 5], we can approximate the H -divergence by a learning algorithm todiscriminate between pairwise source domains.
For example, with source domains D is and D js , welabel the samples of D is by 1 , and the samples of D js by 0 .
We then train a classifier (e.g. linear
SVM) to discriminate between these two domains.
Given a test error ε of this classifier, the Proxy
A-distance (PAD) defined as 2(1 − 2 ε ) can approximate the H -divergence.
We follow the method from [19, 11, 1] to compute the PAD.
For a pair of source domains, we combinethese domains and construct a new dataset.
This dataset is randomly split into two subsets of equalsize.
One subset is used for training and the other one is used for test.
We train a collection of linear
SVMs (with different values of regularization parameters) on the training set and compute the errorsε of all the SVMs on the test dataset.
The lowest error ε is used to compute the PAD.
Fig.
2a compares the PAD of the raw source domains and the mapped source domains.
Theexperiments are conducted on PACS with AlexNet backbone.
For the raw source domains, we extractfeatures from the baseline model (i.e. the last pooling layer of the AlexNet) to train the linear SVMs,while for the mapped source domains, we use features from our domain-invariant classifier.
In thefigure, each dot represents a pair of source domains (e.g. Art and Photo). It has two values: the
PAD of the source domain pair obtained upon the baseline model ( x axis) and the PAD of the samepair computed upon our framework ( y axis).
All the dots are below the diagonal meaning that the
PAD values of the mapped pairwise source domains are lower than the raw pairwise source domains.
With our framework, the mapped source domains become harder to be distinguished, indicating thatremoving the domain-specific features reduces the distribution difference among source domains.
This also proves that ϵ in the generalization risk bound (Eq. 11) is reduced by our framework.
3.3.2 Source-Target domain divergence
We also investigate the distribution difference between the source domains and the target domain.
Specifically, we measure the domain divergence between the target domain and the closest mixturesource domain ¯ D t to the target domain. To obtain this mixture source domain, as defined in Sec. 2.3,we need to find π i for each source domain D is , so that ¯ D t = (cid:80) Ni =1 π i D is , where 0 ≤ π i ≤ 1 and
Proxy A-distance on baseline 1 .1 .1 .1 .1 .2 .
P r o xy
A - d i s t a n ce o n
L R D
G
A,C
A,P
C,PC,P P,S C,S A,C
A,S C,S A,P P,SA,S A: Art
C: Cartoon P: Photo
S: Sketch (a) 1 .1 .1 .1 .1 .1 .1 .2 .
Proxy A-distance on baseline 1 .1 .1 .1 .1 .1 .1 .2 .
P r o xy
A - d i s t a n ce o n
L R D
G { A,C,P } ,S { C,P,S } ,A { A,P,S } ,C { A,C,S } ,P
A: Art C: Cartoon
P: Photo S: Sketch (b)
[Figure]
[-Ni =
 π i = 1 .-][Equation]
Because π i can be any real value in the interval of [0 , 1] , traversing all values to find thedesired
π i is impossible.
Therefore, We limit the values of π i to the set of { 0 , 0 . 1 , 0 . 2 , · · · , 0 . 9 , 1 } (11 258 values in total), and find the setting of { π i } Ni =1 that can obtain ¯ D t .
We traverse all possible settingsof { π i } Ni =1 and obtain all possible mixture source domains ¯ D = (cid:80) Ni =1 π i D is .
For each setting of{ π i } Ni =1 , we random sample π i n t samples from each source domain D is and concatenate all these 261 samples into a mixture source dataset.
n t is the number of samples in the target domain.
By thisdesign, each mixture source domain has an equal number of samples to the target domain.
Similar to
Sec.
3.3.1, we also train classifiers (i.e. linear SVMs) to discriminate between each mixture sourcedomain and the target domain.
The linear SVMs are trained on image features extracted from thebaseline model.
We then use the test error to compute the PAD between each mixture source datasetand the target dataset.
The mixture source domain with the lowest PAD is the closest mixture sourcedomain ¯ D t to the target domain.
The detailed settings of π i for the closest mixture source domainsto the corresponding target domains on PACS are listed in Table 4.
For convenience, we denote theclosest mixture source domain ¯ D t and the target domain D t together as a source-target domain pair.
[Table]
Sketch
2b shows the PAD of the raw sourcetarget domains and the mapped sourcetarget domains.
Similar to Sec.
3.3.1, forthe raw source-target domains, we extractfeatures from the baseline model to trainthe linear SVMs.
To compute the PADof the mapped source-target domains, weextracted image features from our domaininvariant classifier to train the linear SVMs.
In the figure, each dot represents a sourcetarget domain pair (e.g. {Cartoon, Photo,
Sketch}, Art).
We can see that all the dotsare below the diagonal.
The PAD values ofthe mapped source-target pairs are lower than the raw source-target pairs.
This indicates that thedistribution difference between the source and the target is reduced by our framework.
γ in thegeneralization risk bound (Eq. 11) is therefore reduced.
Removing the domain-specific features fromthe source domains can also reduce the distribution difference between the source and the target.
In summary, our framework can reduce the distribution difference not only among source domains butalso between the source domains and the target domain.
This also demonstrates that our frameworkcan effectively lower the value of the generalization risk bound by reducing ϵ and γ .
Related work
Domain generalization is a challenging task that requires models to be well performed on unseendomains.
One common approach is to learn domain-invariant features among the source domains.
Muandet et al.
[33] propose to reduce the domain dissimilarity by a kernel-based method.
Ghifary etal.
[18] reduce dataset bias by extracting features that are shared among domains with a multi-taskautoencoder network.
Li et al.
[29] utilize the Maximum Mean Discrepancy (MMD) on adversarialautoencoders to align the distributions across source domains.
Li et al.
[30] design an end-to-endconditional invariant deep neural network that minimizes the discrepancy of conditional distributionsacross domains.
Arjovsky et al.
[3] develop Invariant Risk Minimization (IRM) that uses a causalmechanism to obtain the optimal invariant classifier upon the representation space.
Chattopadhyayet al.
[10] propose to learn domain-specific binary masks to balance the domain-invariant anddomain-specific features for the prediction of unseen target domains.
Zhao et al.
[48] proposean entropy regularization method to learn the domain-invariant conditional distributions by usingclassification loss and domain adversarial loss.
Du et al.
[13] develop a probabilistic meta-learningmethod that learns domain-invariant representations with meta variational information bottleneckprinciple derived from variational bounds of mutual information.
Mahajan et al.
[31] assume thatdomains are generated by mixing causal and non-causal features and the same object from differentdomains should have similar representations.
Based on this, they propose a new method called
MatchDG to build a domain-invariant classifier by matching similar inputs.
Rame et al.
[35] matchthe gradients among source domains to minimize domain invariance.
Recent papers demonstrate that CNNs tend to classify objects based on features from superficial localtextures and backgrounds, while humans rely on global object shapes for classification [23, 17].
Toaddress this issue, some methods aim to capture the global object shapes from the images.
Thesemethods are proposed based on the assumption that the local textures and backgrounds are the domainspecific features, and the global object shapes are the domain-invariant features.
Wang et al.
[45]extract semantic representations by penalizing features extracted with gray-level co-occurrence matrix(GLCM) which are sensitive to texture.
Wang et al.
[44] penalize the earlier layers of CNNs fromlearning local representations and make the CNNs rely on the global representations for classification.
Conclusion
In this work, we propose a new approach that aims to explicitly remove domain-specific featuresfor domain generalization.
Following this approach, we develop a novel domain generalizationframework that learns domain-invariant features by actively removing domain-specific featuresfrom the input images.
We also experimentally verify the reduced domain divergence among sourcedomains and target domains brought by our approach.
Experiments show that our framework achievesstrong performance on various datasets compared with existing domain generalization methods.
Despite the advantages of our framework, it has some potential limitations to be further addressed.
We need to train the same number of domain-specific classifiers as the source domains.
When thereare more source domains, more computational resources will be required to train the domain-specificclassifiers.
This may be addressed by designing a novel domain-specific classifier that can learndomain-specific features of multiple source domains simultaneously.
Another limitation of ourframework is that it cannot remove the domain-specific features of the unseen target domain.
Thesedomain-specific features should also be removed since they would negatively affect classificationperformance.
For example, our framework performs slightly worse than the baseline when Photo isthe target domain (as shown in Table 1).
This may be because Photo contains rich domain-specificfeatures compared with the source domains, and our framework would make incorrect predictionsdue to these domain-specific features.
Besides, this result also shows that the source domain-specificknowledge is useful for Photo.
As the target domain is not available during training, how to removedomain-specific features from the target domain and whether to remain the domain-specific featuresof the source domains will be challenging issues to research. One possible future direction maybe to remove the domain-specific features in a latent feature space. To achieve this, the frameworkmay need to be adjusted, including the domain-specific classifiers and the domain-invariant classifier. Also, the encoder-decoder network incurs extra computational overhead, but performing on a latentspace may have the benefit that we may no longer need the encoder-decoder network and the overallframework can be computationally more efficient.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Section 5.
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[No] .
The code will
be released and indicated as a URL after paper review.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] In the Appendix.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] In the Appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] In the Appendix.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] In the Appendix.
(b) Did you mention the license of the assets?
[Yes] In the Appendix.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes] In the Appendix.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]