Dual-stream Feature Augmentation for Domain Generalization

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-31 06:31:11

Abstract: Domain generalization (DG) task aims to learn a robust model from source domains that could handle the out-of-distribution (OOD) issue. In order to improve the generalization ability of the model in unseen domains, increasing the diversity of training samples is an effective solution. However, existing augmentation approaches always have some limitations. On the one hand, the augmentation manner in most DG methods is not enough as the model may not see the perturbed features in approximate the worst case due to the randomness, thus the transferability in features could not be fully explored. On the other hand, the causality in discriminative features is not involved in these methods, which is harm for the generalization of model due to the spurious correlations. To address these issues, we propose a Dual-stream Feature Augmentation (DFA) method by constructing some hard features from two perspectives. Firstly, to improve the transferability, we construct some targeted features with domain related augmentation manner. Through the guidance of uncertainty, some hard cross-domain fictitious features are generated to simulate domain shift. Secondly, to take the causality into consideration, the spurious correlated non-causal information is disentangled by an adversarial mask, then the more discriminative features can be extracted through these hard causal related information. Different from previous fixed synthesizing strategy, the two augmentations are integrated into a unified learnable model with disentangled feature strategy. Based on these hard features, contrastive learning is employed to keep the semantics consistent and improve the robustness of the model. Extensive experiments on several datasets demonstrated that our approach could achieve state-of-the-art performance for domain generalization.

Main paper: 
 related work: 
 Domain Generalization. The goal of DG task is to learn the generalizable representations from source domains to ensure stable performance to unknown target domain. Existing methods can be roughly divided into domain-invariant or causal-related feature learning [3, 26, 28, 29] , data augmentation [10, 18, 44, 46] and other learning strategies, such as meta-learning [3, 7, 49] and contrastive learning [8, 17, 47] . Domain-invariant representation learning has become an important method in DA and DG since [13] was proposed. This method facilitates the model in learning domain invariant features through min-max adversarial training between the semantic feature extractor and domain discriminator. [3] also employed a dual path strategy, integrating domain-invariant and domain-specific encoders, similar to our approach. However, they trained two domain classifier for two encoder respectively, which still constitutes an adversarial training process. In recent years, there has been increasing interest in investigating domain generalization from the causal perspective. [28, 42, 45] derived causal information that truly determine the category label from the statistical relationship between the sample and the label. [28] analyzed the three fundamental properties that causal factors should satisfy, thereby achieving the objective by ensuring the learned representations comply with these three properties. However, this may lead the model to learn some domain-specific information from the source domains. An adversarial mask is employed to disentangle spurious correlated non-causal information as in [28] . However, in our work, the mask is applied to domain-invariant features to avoid negative impacts. [17] employed a domain-aware contrastive learning that aims to minimize the distance between stylized and original feature representations. [47] proposed an proxy-based contrastive learning approach. This method used proxies as the representatives of sub-datasets and managed the distance between features and proxies, thereby enhancing the robustness against noise samples or outliers. [8] generated domain-invariant paradigms for each instance and then conducted contrast learning between the features of image instances and their paradigms. In our work, we apply supervised contrastive learning strategy to dual stream augmented features and domain-invariant features. This enables the model to eliminate potential stylistic information and non-causal information inherent in the domain-invariant features, thereby enhancing the model's generalization ability. Data Augmentation. Data augmentation techniques for Domain Generalization (DG) can be broadly categorized into image generation [53, 55] , image transformation [41, 46] , and feature augmentation [48, 56] . However the offline two-stage image generation training procedure is complex, as both training a generative-based model and inferring it to obtain perturbed samples present significant challenges. [46] perturbed the style of a sample through linear interpolation between the Fourier spectrum amplitude components of the sample. However, it randomly selected the exchange sample and ratio. [48] employed Wavelet Transforms to decompose the features into high and low frequencies. [27, 41] achieved feature style transformation by executing a series of processes on the lowfrequency component of features. The statistical properties [33] of the feature maps can represent stylistic information as they capture visual properties, [18, 44, 52, 56] achieved style transformation by perturbing statistics of features. However, these methods directly interfere with feature statistics and often fail to maintain semantic consistency. Very recently, [22] is proposed to explicitly enforce semantic consistency preserving class-discriminative information. It generated learnable scaling and shifting parameters for features to enhance domain transfer from the original ones and this idea is very similar with ours. However, it is essentially still a random augmentation method, while our method aims to generate targeted features. In our method, we construct hard augmented features through DFA, enhancing the generalization capacity of the model. Not only domain style transformation but also causal related information augmentation is implemented in our work. 
method: 
 The source domains D s and target domain D t share the same label space in DG task. Each source domain consists of D s = {(𝑥 𝑖 , 𝑦 𝑖 )} 𝑁 𝑖=1 , in which 𝑥 represents sample and 𝑑 represents label. In our method, we use dual path feature disentangle module to obtain domain-invariant features and domain-specific features. Then the adversarial mask module is introduced, the potential causal information is mined through it to disentangle spurious correlated non-causal information among domain-invariant features. Finally, we present the dual-stream feature augmentation.Domain-invariant features refer to the shared semantic characteristics across multiple domains, which remain consistent despite domain shifts. Features that cannot be distinguished by the domain classifier are considered effective domain-invariant features. Ensuring that the domain classifier can accurately identify domain features is crucial. Most methods are updating both the domaininvariant encoder 𝐹 𝐼 and the domain classifier 𝐶 𝑑 together. However, training domain classifier with domain-invariant features which do not contain domain-specific information could not guarantee its effectiveness. Therefore, we propose a dual path feature disentangle module, which ensures the accuracy of the domain classifier by leveraging an extra domain-specific encoder. The proposed method consists of a domain-invariant encoder 𝐹 𝐼 , a domainspecific encoder 𝐹 𝑆 and a domain classifier 𝐶 𝑑 . To achieve an optimal domain classifier, we conduct the training of the domainspecific encoder and the domain classifier as a 𝑘 classification task, as defined by Eq 1, where 𝑑 𝑖 represents domain label and 𝑘 represents the number of source domains.EQUATIONRegarding the domain-invariant features 𝑓 𝐼 , they are passed through the domain classifier 𝐶 𝑑 to obtain the domain classification probability 𝑃 𝑓 𝐼 . The domain-invariant encoder 𝐹 𝐼 is then updated by Eq 2. Noteworthily, to force the features not containing domain specific information, instead of the cross entropy loss, we use the mean square error (MSE) loss to make the classification probabilities as smooth as possible. The reason is that the domain classifier is supposed to be impossible to distinguish domain-invariant features.L 𝑖𝑛𝑣 𝑑𝑐 = (𝑃 𝑓 𝐼 - 1 𝑘 ) 2 (2)To ensure that the features are causally sufficient and contain more potential causal information, the adversarial mask module [28] is employed to achieve the goal. We aim to categorize the feature dimensions into superior dimensions, which are related to causal information, and inferior dimensions, which lack sufficient causal information and exhibit spurious correlations with the labels. Obviously, the superior dimensions of features have stronger relevance to the semantics than the inferior. Through the adversarial training between two classifiers, the two kinds of features can be separated. Specifically, a neural network 𝑀 is built, by using derivable Gum-belSoftmax to sample the mask 𝑀 (𝑥) . Through multiplying the domain-invariant features with the resulting masks 𝑀 𝑠𝑢𝑝 = 𝑀 (𝑓 𝐼 ) and 𝑀 𝑖𝑛𝑓 = 1 -𝑀 (𝑓 𝐼 ), we can obtain superior and inferior features, respectively, and then feed them into two different classifiers 𝐶 1 and 𝐶 2 . The optimization process between encoder, classifier and mask is an adversarial learning process. On the one hand, two classifiers and encoder are optimized by cross-entropy loss, so that they can mine more semantic information, as shown in Eq 3. On the other hand, the mask is optimized through adversarial training by maximizing the classification loss of the inferior dimensions, as shown in Eq 4, to better distinguish superior and inferior dimensions. Adv MaskFigure 3 : The framework of DFA. We first generate domain-invariant features and domain-specific features by dual path feature disentangle module, and employ adversarial mask module to disentangle spurious correlated non-causal information from domain-invariant features. We combine superior features with domain-specific information and non-causal inferior information by special strategy respectively to achieve dual-stream feature augmentation. At last, contrastive learning is adopted to the augmented features and domain-invariant features. The dashed lines represent that the gradient is detached.EQUATIONThe overall loss of the adversarial mask module is depicted in Eq 4 and Eq 5.EQUATIONL 𝑐𝑙𝑠 = L 𝑖𝑛𝑓 𝑐𝑙𝑠 + L 𝑠𝑢𝑝 𝑐𝑙𝑠(5)In this section, we will introduce the two types of feature augmentation methods respectively. Assuming that each domain contributes 𝑛 samples to a batch and there are 𝑘 source domains in total, the batch size is calculated as 𝐵 = 𝑛 × 𝑘. To construct domain related hard features, the superior dimensions of domain-invariant features and the most distinct style from the domain-specific features in other domains should be selected.To achieve this goal, information entropy is leveraged as the criterion for evaluating style features, as illustrated in Eq 6. Lower information entropy indicates a more distinct style information. To ensure augmentation diversity, we select one sample from each domain sequentially, using 𝑘 samples as a group to implement the aforementioned augmentation. And there are 𝑛 groups within a single iteration. The chosen domain-specific features and disentangled superior dimensions features are merged via the concatenation operation. Then subsequently the feature dimension is reduced through a fully connected layer, as shown in Eq 7. The [•, •] is a concatenation operation and FC is a fully connected layer. 𝑓 ′ 𝑆 represents the domain-specific features that belong to other domains in a group.𝐼𝐸 (𝑋 ) = -𝑃 (𝑥)𝑙𝑜𝑔(𝑃 (𝑥))𝑓 𝐷𝑅-𝐴𝑢𝑔 = FC([𝑓 𝑠𝑢𝑝 , 𝑚𝑖𝑛(𝐼𝐸 (𝑓 ′ 𝑆 ))]) (7) Intuitively, the semantic information of domain related hard features aligns with domain-invariant features, and domain-invariant features inevitably retain some aspects of their original domainspecific information. To eliminate potential domain-specific information in domain-invariant features, the supervised contrastive learning is applied to domain-invariant features and domain related hard features, as shown in Eq 8. By drawing the positive samples close and the negative samples separated in the feature space, the transferability of the domain-invariant features can be enhanced.EQUATION3.3.2 Causal related feature augmentation. Due to the limitations of the dataset and the insufficient diversity of samples, the model would inevitably learn some spurious correlated non-causal information when capturing the statistical relationship between samples and labels. In our framework, although we employ adversarial mask module to disentangle the spurious correlated noncausal information of domain-invariant features, it can not entirely eliminate the non-causal information due to the lack of diversity. Therefore, we propose the causal related feature augmentation to create causal related hard features to enhance the diversity. To construct cross-class causal related hard features, we select a cross-class non-causal information, which is in the form of inferior dimensions of domain-invariant features, for each disentangled feature. Intuitively, the spurious correlated non-causal information in one class may also exhibit a degree of spurious correlation with its similar categories. With this insight, the class with the greatest classification probability excluding its label class is selected as objective for non-causal information selection. Similar with the domain related feature augmentation, information entropy is served as the criterion for selecting non-causal information among objective classes. As illustrated in Eq 6, lower information entropy indicates a more certain spurious causal correlation. Specifically, to mitigate the impact of varying domain information, we implement feature augmentation within one domain. Within a specific domain, the information entropy criterion is leveraged to select a non-causal information feature for different categories, resulting in the selection of 𝐶 features for 𝐶 classification tasks. With 𝑘 source domains in total, the overall number of selected non-causal information features is 𝑘 × 𝐶. Subsequently, for each disentangled feature, we select the non-causal information feature corresponding to its target class from the 𝐶 non-causal features in its source domain. The chosen non-causal features and disentangled features are then merged via the concatenation operation and reduce the feature dimension through a fully connected layer following the above section operation, as shown in Eq 9. 𝑓 ′ 𝑖𝑛𝑓 represents the inferior dimensions of domain-invariant features that belong to the objective selection class within a domain.EQUATIONThe domain-invariant features contain the same causal information as the causal related hard features. To encourage the domaininvariant features to disregard the non-causal information, we employ supervised contrastive learning between domain-invariant features and causal related hard features, as shown in Eq 10.EQUATIONThe overall loss of the dual-stream feature augmentation is depicted in Eq 11.EQUATIONThe overall training process is composed of three components.  
experiments 4.1 dataset: 
 To verify the effectiveness of the proposed method, we evaluate our method on four public datasets, which cover various recognition scenes. PACS [24] is a public object recognition dataset which has large discrepancy in different domains. It contains 999,1 images from four domains (Art-Painting, Cartoon, Photo and Sketch), and in each domain, it contains 7 categories. For fair comparison, we follow the original training-validation split provided by [24] . OfficeHome [40] is a large public dataset with 4 domains, and each domain consists of 65 categories. The four domains are Art, Clipart, Product and Real-World. It contains 15,500 images, with an average of around 70 images per class. Following [28] , we randomly split each domain into 90% for training and 10% for validation. VLCS [14] is a mixture of different datasets, named as VOC2007 [11] , LabelMe [34] , Caltech101 [12] and SUN09 [9] . Each domain contains 5 categories. Following [8] , we randomly split 80% for training and 20% for validation. TerraIncognita [2] is a very large dataset includeing 24,778 photographs of wild animals, which are divided into 10 categories. It contains 4 camera-trap domains: L100, L38, L43, L46. ImageNet pretrained on ResNet [15] is used as our backbone. We train the network with SGD, batch size of 16 and weight decay of 5e-4 for 50 epochs. The initial learning rate is 0.001 and decayed by 0.1 at 80% of the total epochs. For all datasets, images are resized to 224 × 224. The standard augmentation protocol in [4] is followed, which consists of random resized cropping, horizontal flipping and color jittering. We also adopt the Fourier data augmentation as in [46] and construct different domain-specific encoder for different source domain. Following the commonly used leave-onedomain-out protocol [25] , we specify one domain as the unseen target domain for evaluation and train with the remaining domains.The parameter 𝜆 𝑖𝑛𝑣 of the domain classifier loss is set to 1 and use a sigmoid ramp-up strategy [31] with a length of 5 epochs following [28] . To promote greater stability during training, we apply identity operations to the mask throughout the initial five epochs as [28] . The parameter 𝜆 𝑐𝑙 of the contrastive learning loss is set to 0.001 after the initial five epochs. Inspired by [19] , the temperature parameter 𝜏 of ℓ 𝑐𝑙 is set to 0.07.Results on PACS is shown in Table 1 . We compare our method with previous SOTA methods. Our method surpasses CIRL [28] by 0.48% on ResNet18 and 0.28% on ResNet50, respectively. This 53.0 43.0 57.9 40.4 48.6 SWAD [6] 55.4 44.9 59.7 39.9 50.0 PCL [47] 58.7 46.3 60.0 43.6 52.1 DFA(ours) 59.9 50.2 57.0 42.8 52.5improvement is attributed to learning causal information from domain-invariant information, thereby excluding causal-related but domain-specific information. Furthermore, our method outperforms data augmentation methods such as FSR [44] and FFDI [41] , because we encompasses not only feature stylization augmentation but also causal-related feature augmentation. Specifically, compared with CCFP [22] , which also adopt feature augmentation, DFA surpasses CCFP by 2%. Through dual-stream feature augmentation, both the transferability and discriminability of features are enhanced. Our method achieves the best performance, achieving an average accuracy of 86.80% on ResNet18 and 90.40% on ResNet50.Results on OfficeHome is shown in Table 2 which illustrates that DFA outperforms data augmentation methods like FACT [46] , FSR [44] and FFDI [41] . However, the impact of DFA on ResNet18 is limited due to the image number per category is small in this dataset and the data style is similar to its pretrained dataset ImageNet with a small domain gap. In such scenarios, some domain style information may enhance the classification results. On ResNet50, DFA is 2.9% higher than mDSDI [3] method, and 0.5% higher than PCL [47] method. Results on VLCS is shown in Table 3 . we use ResNet18 as the backbone and our DFA demonstrates superior performance, outperforming CIRL [28] by an average of 0.46%, and surpassing FSR [44] by an average of 2.74%. Results on TerraIncognita is shown in Table 4 and ResNet50 is employed as the backbone. Our DFA exhibits superior performance, surpassing mDSDI [3] by an average margin of 4.4%. DFA outperforms PCL [47] by an average of 0.4% which is a robust contrastive learning method. Based on the above results from four benchmarks in DG task, DFA outperforms other data augmentation methods, particularly in situations with large domain gaps. DFA achieves the feature augmentation by considering both domain-specific information and causally correlated information, thereby the generalization capability of the model is improved. 
discussion: 
 Ablation Study. We conduct ablation studies to demonstrate the significance of each module in Table 5 . "DFD" and "AdvM" represent dual path feature disentangle module and adversarial mask module, respectively. "DR" and "CR" represent domain-related feature augmentation and causal-related feature augmentation, respectively. We employ ResNet18 as the backbone and train on the PACS dataset. Firstly, we discuss the ablation study of the baseline ( corresponds to model3 in the table) which represents the feature disentanglement framework without the hard feature perturbation. Comparing model3 with model1 and model2, it is obvious that the performance of combining both Adversarial Training and Adversarial Mask is much better. This observation suggests that for DG problems, it is insufficient to learn only domain-invariant features or causal features. Rather, considering causal information within domain-invariant features can directly improve model performance. Additionally, the performance enhancements seen in Model4 and Model5 indicate that the two types of feature augmentation methods we proposed can help the model concentrate on hard features, thereby improving the model's ability to discriminate hard features. Finally, based on the baseline, the DFA achieves the SOTA result of 86.80%, demonstrating that the two types of feature augmentation methods further enhance the transferability and discriminability of features.Analysis with GradCAM. In Figure 4 , we visualize the attention maps of the last convolutional layer to verify the efficacy of causal related feature augmentation. The second row presents the baseline, while the last row demonstrating the efficacy of causal related feature augmentation. It is evident that, despite the baseline achieving relatively satisfactory test results, it still encounters challenges with samples that have spurious correlations. It fails to capture the causal information, instead focusing on non-causal information, as demonstrated in Figure 4 . These results suggest that causal related feature augmentation can effectively enhance the model's ability to identify the causal information of samples, consequently strengthening discriminability of features.Analysis of Feature Statistics. To confirm that domain related feature augmentation can effectively mitigate the effects of domain shift, we visualize the feature statistics distribution based on Mixstyle, baseline and DFA. Compared with Mixstyle [56] , the baseline successfully learns domain-invariant information, exhibiting minimal shifts in feature statistics and our feature augmentation clearly mitigates the domain shift between different domain features, indicating a higher purity of domain-invariant features.Confusion Matrix. We have plotted confusion matrix for our baseline and DFA, as illustrated in Fig. 7 . We employ ResNet18 as backbone and train on the PACS dataset.It can be obviously found that in the art and cartoon domains, the baseline still has some incorrect classifications due to non-causal information and domain shift. In contrast, DFA displays a significant reduction in classification errors. This evidence suggests that DFA is capable of eliminating such spurious correlations in samples and paying attention to domain-invariant and causally related information, thereby enhancing the model's generalization ability.Visualization of Features. We employ t-SNE [37] to display the visualization results of features extracted by the semantic feature extractor, as depicted in Fig. 6 . From Fig. 6(a) , where different colors denote different classes, it becomes clear that although the baseline can distinguish each category in the feature space, it still struggles to differentiate samples with similar semantics. This challenge is indicated by a mixture of points from different class labels 𝑑𝑐 and L 𝑐𝑙 converging, indicating that our method provides a stable end-to-end framework. 
conclusion: 
 In this paper, a dual stream feature augmentation is proposed based on the disentanglement framework. Previous work always applied random perturbations on style, however, they do not exploit potentialities of feature transferability. Differently, on the one hand, we construct domain related hard features to explore harder and broader style spaces while preserving semantic consistency. On the other hand, considering that the spurious correlated non-causal information can harm the discriminability of model, the causal related hard features are also constructed to better disentangle the non-causal information hidden in domain-invariant features, thereby improving the generalization and robustness of the model. Trough dual-stream feature augmentation based on a stable feature disentanglement framework, we successfully learn causal related domain-invariant features, and a variety of experiments demonstrate the effectiveness of our method. In the future, we will try to integrate our work with the challenging multimodel learning task. 
: 
 (a) The dashed areas represent the mathematical statistical relationship between each domain and labels. Obviously, the areas not only include the shared part, but also contains the specific parts. (b) The dashed area represents the domain invariant information across multiple source domains. However, the spurious correlation information still exists in it. (c) Our motivation is to learn domain invariant features that have causal relationships with the labels.scenarios often pose challenges such as non-stationary and unknown distributions in testing data. To address distribution shifts between training and testing data, the domain generalization task has emerged. This task aims to make the model robust and generalized across multiple source domains, enabling its application in unknown target domains. In order to obtain the generalizable and accurate features in DG task, the criteria of transferability and discriminability are both important. As shown in Fig 1(a) , due to the existence of domain shift, the mathematical statistical relationship between features and labels are different in different domains. Even the model has high discriminability in source domains, it can not work well in target domains due to domain shift. However, only concerned about the transferability is not enough. Shown in Fig 1(b) , the dashed area refers to the domain-invariant information learned from multiple domains. Although leveraging domain adversarial learning can get good transferability of features, the model only focuses on the domain shared information, which maybe harm the final downstream tasks. e.g., some non-causal information that has spurious correlations with labels can not be distinguished. Then the model could not generalize well to unseen domains. Based on this, in order to relieve the above issue, we want to achieve the goal shown in Fig 1(c ), which could not only eliminate the spurious correlated non-causal information, but also exploit the domain invariant features with sufficient causality. Data augmentation has been demonstrated effective in DG task recently. Generally speaking, these augmentation methods keep the semantics consistent and modify the style of samples to enhance the diversity. This strategy goes against domain shift and makes the model pay more attention to features that are invariant to domain transfer. If the model could fully focus on capturing the statistical dependency between the semantic information and the corresponding labels, it could eliminate bias toward a particular domain distribution. According to the Empirical Risk Minimization (ERM) principle, to improve the generalization capability of a model, an effective way is to optimize the worst-domain risk over the set of possible domains. However, despite the performance could be promoted by generating samples that are created to approximate the worst case across the entire family of domains, it is hard to generate "fictitious" samples in the input space without losing semantic discriminative information. Moreover, previous methods always adopt the two-stage data perturbation training procedure, and the perturbed samples can not achieve the selfadaptation with the different samples. Fourier transform [46] is a well-known data augmentation manner and obtains competitive results. Usually, in order to avoid the semantic changes, domain transfer is achieved by adding random noise to the Fourier spectrum amplitude components of the sample, then the new data augmentation can be generated. However, this random way may induce unpredictable alterations in the image style. Minor disturbances might have no significant impact on the domain style, rendering the style transformation ineffective. Conversely, substantial perturbations could distort the image style, potentially affecting its semantics and introducing label noise.Based on this, we aim to achieve the data augmentation by hard perturbing features without changing the semantics. In order to fully explore the generalization boundaries and avoid the semantic level collapse, instead of samples, the data augmentation in our method is performed on the feature level. We propose to perturb the hard features based on a feature disentanglement framework, as shown in Fig 2 . On the one hand, to obtain the transferability, we aim to construct the consistent semantic augmented features with another domain information. As illustrated in Fig 2 left , the domainspecific information with the most abundant style attributes is selected to construct domain related hard features. In information theory, the entropy is an uncertainty measure which can be leveraged to quantify the domain style. However, only rely on reducing the domain difference to improve the generalization is not enough, although the feature disentanglement could guarantee the semantics not be changed, it does not involve the spurious correlation and non-causal information in the features. In DG task, causality is an important factor for the discriminability. On the other hand, to improve the reliablity of the statistical dependence, the spurious non-causal correlations should be eliminated and the invariant causal correlations should be mined. Based on the semantic features of disentanglement, we propose to construct the causal related hard features. As shown is Fig 2 right , the non-causal information with similar labels exhibits spurious correlation with semantics, which could be used to construct causal related hard features. These features consist of the domain-invariant semantics and the spurious correlated non-causal information from another class. The causal and non-causal related information in our method can be separated by an adversarial mask. With the help of the two stream hard features, our model could fully explore the causal factor based on the domain invariant features, thereby the transferability and discriminability of features can be fully preserved.In this paper, we integrate the two feature augmentation manner into one unified framework and propose a dual stream feature augmentation based on the feature disentanglement framework. In the framework, domain-invariant causal features are obtained through the feature disentanglement strategy with the help of domain related and causal related hard features. To keep the semantics consistent, contrastive learning is leveraged to dual-stream augmented hard features respectively. Noteworthily, the perturbed features are created during the training process, therefore the online features can adapt with the different inputs. The contributions of our work are as follows:• We point out the disadvantages of present data augmentation methods, and propose a domain related hard feature perturbation strategy with semantic consistency based on an end-to-end stable disentanglement framework, thereby improving the transferability of features. • To fully explore the discriminability in generalized features, the causality is considered in our method. Causal related hard features are created with the disentanglement framework, thus eliminating the underlying non-causal information hidden in features. • We conduct extensive experiments on several public benchmarks, which clearly demonstrate the effectiveness of our approach. 
 
 Appendix:  


Decision: The reviews recognize the paper's innovative Dual-stream Feature Augmentation (DFA) framework, which enhances domain generalization by integrating domain-relevant and causality-aware feature generation. This approach effectively addresses spurious correlations and improves feature transferability, as demonstrated by extensive experiments across various datasets. While some reviewers noted minor issues with experimental details and methodological clarity, these concerns are outweighed by the paper's significant contributions and novel strategies. Therefore, despite the need for minor refinements, the paper merits acceptance.

Reviews: review: paper_summary: paper_summary: The paper presents a method aimed at constructing hard features from two perspectives: domain-related and causal-related, to improve the model's transferability and discriminability. It integrates these augmentations into a unified learnable model with a disentangled feature strategy and employs contrastive learning to maintain semantic consistency and robustness.
strength_weakness: strength_weakness: 1. Introduces a novel dual-stream augmentation strategy for improving generalization across domains.

2.  Addresses the issue of spurious correlations by disentangling causal and non-causal information.

3. Provides extensive experimental results on various datasets to validate the effectiveness of the proposed method.
limitations: limitations: 1. The experimental section is somewhat incomplete, with the following issues:
   - The baselines for comparison are not explained.
   - The comparative work lacks the most recent research efforts, with most being from 2022 and earlier.
   - There are no results for VLCS using the ResNet50 backbone.

2. The choice of backbone: Why are methods still using ResNet18 as the backbone for comparison? For recent DG tasks, isn't the default backbone usually ResNet50 or Vision Transformer?

3. Why is the weight of $\lambda_\text{cl}$ so low? Does $l_\text{cl}$ really play a role?
paper_summary: paper_summary: Traditional DG methods often fall short because they do not adequately explore feature transferability due to the randomness of feature augmentation, and they fail to account for causality in discriminative features, which leads to issues with spurious correlations.

The DFA method addresses these problems through two main strategies. First, it improves feature transferability by creating targeted features with domain-related augmentations, guided by uncertainty to simulate hard cross-domain scenarios. Second, it incorporates causality by using an adversarial mask to disentangle spurious, non-causal information, allowing for the extraction of more discriminative, causal-related features.

This approach differs from previous methods by integrating the two types of augmentations into a unified, learnable model with a disentangled feature strategy. Additionally, contrastive learning is employed to maintain semantic consistency and enhance the model's robustness. Extensive experiments on several datasets have shown that the DFA method achieves state-of-the-art performance in domain generalization, demonstrating its effectiveness in overcoming traditional DG challenges.
strength_weakness: strength_weakness: The authors focus on the domain related hard features and make adversarial augmentations to boost the performance of DG.
limitations: limitations: The computational cost may be larger than other methods.
paper_summary: paper_summary: This paper proposes two augmentations from the perspectives of transferability and causality respectively and integrated them into a unified learnable model to solve existing limitations of DG methods. Experiments on classical DG datasets its effectiveness.
strength_weakness: strength_weakness: 1.	The method achieves SOTA on several classical datasets as reported in the paper.
2.	The algorithm design makes sense, and the specific implementation demonstrates thoughtful design elements that are interesting.
3.	Figure 1 is clear.
limitations: limitations: 1. The author is attempting to explain their method in detail. However, some expression in "Method" section is still rather unclear. For example, why “the semantic information of domain related hard features aligns with domain-invariant features”?
2. The experiments are not comprehensive enough. For example, VLCS is only conducted with ResNet18 while TerraIncognita only have ResNet50 results, which is not a mainstream implementation in DG.
3. Some demonstrations are not clear enough. The caption of Figure 2 is confusing. Figure 4 may need to include more attention maps about essential baselines other than ERM. The analysis regarding the confusion matrix and Figure 7 in discussion seems to offer limited value.
paper_summary: paper_summary: To address the current issues in domain generalization, such as insufficient exploration of extreme perturbed features and the neglect of causal relationships in discriminative features, the paper proposes a Dual-stream Feature Augmentation framework. The framework improves the model's generalization ability to unseen domains by integrating domain-relevant and causality-aware hard feature generation.
strength_weakness: strength_weakness: - Proposing a novel domain-relevant hard feature perturbation strategy. The strategy is based on an end-to-end stable disentanglement framework, which not only maintains the semantic consistency of the data but also effectively enhances feature transferability. This approach offers new insights for improving model generalization in unseen domains.
  
- Exploring and enhancing the distinguishability of generalized features by introducing causality. By creating causal-related hard features through the disentanglement framework, it eliminates the underlying non-causal information hidden in the features, thereby improving adaptability to complex data environments.
  
- Extensive experiments conduct on multiple public benchmarks clearly demonstrate the effectiveness of the proposed method.
limitations: limitations: - In section 3.3.1, the authors state "Lower information entropy indicates a more distinct style information" and in section 3.3.2, it states "Lower information entropy indicates a more certain spurious causal correlations". Both sections use information entropy to measure style differences and causal spuriousness across different domains. Could the authors provide a more explicit theoretical basis and explanation for the direct association between low information entropy and the uniqueness of style or spuriousness of causal relationships?
  
- What baseline or threshold is "Lower" in "Lower information entropy" relative to? If there is a baseline or threshold, are the standards for using information entropy consistent across both instances? Could authors provide a detailed explanation.
  
- Although the authors conduct ablation experiments to validate the innovations proposed in the paper from a comprehensive performance perspective, the related analysis is lacking. It is recommended that the authors enhance the analysis and discussion in the experimental section.
  
- References [24] and [25] are the same document. Please check for citation errors in the references.
  
- It is suggested to add more details to Figure 3, such as "CL"