Domain-Agnostic Crowd Counting via Uncertainty-Guided Style Diversity Augmentation

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 14:03:17

Abstract: Domain shift poses a significant barrier to the performance of crowd counting algorithms in unseen domains.
While domain adaptation methods address this challenge by utilizing images from the target domain, they become impractical when target domain images acquisition is problematic.
Additionally, these methods require extra training time due to the need for fine-tuning on target domain images.
To tackle this problem, we propose an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method, enabling the crowd counting models to be trained solely on the source domain and directly generalized to different unseen target domains.
It is achieved by generating sufficiently diverse and realistic samples during the training process.
Specifically, our UGSDA method incorporates three tailor-designed components: the Global Styling Elements Extraction (GSEE) module, the Local Uncertainty Perturbations (LUP) module, and the Density Distribution Consistency (DDC) loss. 
The GSEE extracts global style elements from the feature space of the whole source domain.
The LUP aims to obtain uncertainty perturbations from the batch-level input to form style distributions beyond the source domain, which used to generate diversified stylized samples together with global style elements. 
To regulate the extent of perturbations, the DDC loss imposes constraints between the source samples and the stylized samples, ensuring the stylized samples maintain a higher degree of realism and reliability.
Comprehensive experiments validate the superiority of our approach, demonstrating its strong generalization capabilities across various datasets and models.
Our code will be made publicly available.

Main paper: 
 related works 2.1 crowd counting: 
 Fully-Supervised Crowd Counting. After the initial proposition of density learning methods by Lempitsky et al. [25] , the crowd counting research has pivoted from detection-based [13] and regressionbased [2, 48] paradigms to methods reliant on density maps. Owing to the physical law of foreshortening, which dictates that objects appear smaller as they recede into the distance, the generated Gaussian density maps may confront significant scale variations. One type of approach commonly adopted to address this issue involves refining network design for more accurate density map estimation. Specifically, some works [4, 45, 57, 62] employ multi-column network architectures to learn feature information across diverse scales. Some studies endeavor to utilize scale-selection [9, 15, 43, 46] techniques or learn to scale [54, 55] methodologies to mitigate the challenges posed by scale variations. Some methods leverage attention mechanisms [1, 30, 32, 41] or deformable convolution [14, 64] to enhance feature representational capacity. Another type of approach elevates the accuracy of crowd counting by innovating how to generate better ground truth density maps. Specifically, Zhang et al. [60] generate density maps by employing perspective normalization to generate perspective maps, which are then amalgamated with the center positions of pedestrian heads. MCNN [62] introduced geometry-adaptive kernels as a replacement for a fixed Gaussian kernel. Wang et al. [50] try to employ attention mechanisms for the adaptive fusion of density maps generated from distinct, predefined Gaussian kernels. Huang et al. [18] and ADMAL [7] propose a method for adaptively generating density maps based on the spatial features of the objects. However, most of these works neglect the domain shift that often occurs between training and testing scenarios in many real-world applications, leading to a significant degradation in model performance. Domain Adaptive Crowd Counting. To counteract the potential domain shifts, Domain Adaptation (DA) techniques have been introduced into crowd counting [7, 36, 50, 67, 68] . Within the DA setting, the network requires target domain images for fine-tuning. [7, 12, 16, 71] all employ adversarial learning at the feature level to force the network to learn domain-invariant features. There are other approaches [10, 50, 51] that deploy adversarial learning techniques at the image level. The crux of this type of method lies in employing the Generative Adversarial Network (GAN) [70] principle to transfer the stylistic attributes of source domain images to the target domain, thereby diminishing the domain shift between the source and target domain. Apart from adversarial learning approaches, Liu et al. [36] use uncertainty estimation to facilitate self-supervised learning in target domain, thereby accomplishing fine-tuning purposes. DAOT [69] proposes a domain-agnostically aligned optimal transport strategy that aligns domain-agnostic factors and managing outliers.Different from domain adaptation methods, which can access both source domain and target domain images, domain generalization methods are limited to obtaining data solely from the source domain. A straightforward approach is to enhance the diversity of source domain data through data augmentation [17, 27, 56, 58] . SHADE [63] proposes a style hallucination module to generate new style-diversified data to improve generalization ability and relieve the model from overfitting in training domains. DSU [28] proposes to use the feature statistics of the training data to improve the generalization ability of model. Another prevalent approach is to leverage the meta-learning technique [8, 23, 26] to align the distribution of the source domain by learning domain-invariant representations. The essence of this category of work is to divide the source domain into meta-train and meta-test sets, thereby simulating the domain shift between the source and target domains. Building on MLDG [26] , DGCC [8] employs meta-learning technique in conjunction with domain-invariant and -specific crowd memory modules to achieve domain generalization in crowd counting. Unlike the DGCC method, we propose an uncertainty-guided style diversity augmentation method that generates a diverse set of samples to prevent the network from overfitting to the source domain. In addition, compared to DGCC, our method introduces no modifications to the network architecture and has no impact on inference time. To our best knowledge, this is the first attempt to address domain generalization issues in crowd counting through data augmentation techniques.Encoder 𝐵×𝐻×𝑊×𝐶 ••• Intermediate Features ••• 2𝑁×𝐶 Feature sta-cs 𝑉 ! FPS Encoder Sampled basis elements ••• 2𝐵×𝐶 2𝐵×𝐶 ••• Source style information C ℒ !!" ℒ #$% ℒ #$% 
methodology 3.1 framework and preliminary concepts: 
 Setting and Overview. Assuming that we have an observable source domainD 𝑠 = {(𝑥 𝑠 𝑖 , 𝑦 𝑠 𝑖 )} 𝑁 𝑖=1, where 𝑁 is the number of images in domain D 𝑠 and (𝑥 𝑠 𝑖 , 𝑦 𝑠 𝑖 ) denotes the image-label pair for the 𝑖-th sample in source domain. Besides, we have 𝑀 unseen target domains {𝐷 𝑡 𝑖 } 𝑀 𝑖=1 . Our objective is for the model 𝜃 , which is well-trained on the source domain D 𝑠 , to also perform well concurrently across these 𝑀 target domains without any finetuning. Moreover, the proposed method is desired to be easily incorporated into crowd counting algorithms without impacting inference speed.To achieve our objective, an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method is proposed, which consists of a Global Styling Elements Extraction (GSEE) module and a Local Uncertainty Perturbations (LUP) module. Thanks to these collaborative modules, the proposed UGSDA method can generate sufficiently stylized samples, thereby enhancing the domain generalization capabilities of crowd counting methods. Furthermore, a Density Distribution Consistency (DDC) loss is proposed to constrain the realism of the density distribution of the stylized samples in the high-dimensional feature space. The overall framework is shown in Fig. 2 . Preliminaries. A model trained on one domain may struggle to adapt to new domains. Data augmentation is a strategy that can expand the amount and diversity of data. Since the network is exposed to data in various styles, it can understand invariances across more data domains [6] . For instance, image flipping enables the network to deal with perspective effects in crowd counting [36] . That is why use style variation methods can help improve the network's generalization ability. However, some image data augmentation methods, such as Rotation, Photometric Transformer, and Mixup [61] , are essentially predefined manipulations or combinations of operations applied to images. These data augmentation techniques fall short in augmenting the variety of data styles and distributions, such as acquiring a new style or altering the distribution ratio of styles within the dataset.In some generative models [19, 22] , it has been found that the use of Adaptive Instance Normalization (AdaIN) [19] allows for adaptation to arbitrarily given styles. The formulation for AdaIN can be expressed as follows:EQUATIONwhere 𝑥 is content input, 𝑦 is style input, 𝜇 ( * ) and 𝜎 ( * ) represent the mean and standard deviation across spatial locations, respectively. Through AdaIN, we can scale the normalized content of input using 𝜎 (𝑦) and shift it using 𝜇 (𝑦), thereby obtaining a new sample that conforms to the style of 𝑦.In domain generalization of crowd counting, we have the content 𝑥 from source domain. To achieve our objectives, the key challenge is to generate real and diverse stylized samples to enhance the model's generalization capability. In Section 3.2, we will discuss how to generate global style elements within the scope of the source domain and Section 3.3 introduces how to generate more out-of-distribution stylized samples with the help of the proposed LUP module.Method. As shown in Fig. 1 , it can be observed that across various crowd counting data sets, there may exist different style transformations, such as night, daytime, various weather conditions, indoor lighting, etc. It is challenging to manually define all possible style information that may be contained in every unseen target domain. Following [63] , we leverage the definition of basis in linear algebra to capture the global style elements within the source domain.To handle out-of-distribution styles, we further employ the Local Uncertainty Perturbation (LUP) module, detailed in Section 3.3.A basis 𝑆 is a set of linearly independent vectors in a vector space 𝑉 , through which every vector in that space can be represented. According to this definition, assuming that the style space is a subspace of a 𝐶-dimensional vector space, it is feasible to represent the entirety of styles using 𝐶 linearly independent basis vectors. However, directly using orthonormal vectors ignores the real styles contained in the source domain, which may yield an excess of spurious styles, adversely affecting the model's performance. It should be noted that with source domain data serving as prior knowledge, sampling basis vectors 𝑆 𝑠 from the style space of the source domain is likely to yield more real new styles. Thanks to the research in style transfer [19, 22] , we can represent the style space 𝑉 𝑠 using the mean and variance of the source domain image features 𝑓 𝑠 = 𝜑 (𝑥 𝑠 ) as follows:EQUATIONwhere 𝜇 𝑐 ( * ) and 𝜎 𝑐 ( * ) represent the mean and standard deviation across channel dimension, 𝑓 𝑠 𝑖 denotes the feature map obtained from processing the 𝑖-th image by the encoder 𝜑 ( * ). Within the defined style vector space 𝑉 𝑠 , we can utilize a sampling method to obtain 𝐶 basis styles every 𝑘 epochs. We sample the combination weight 𝑊 = [𝑤 1 , . . . , 𝑤 𝐶 ] from Dirichlet distribution 𝐵([𝛼 1 , . . . , 𝛼 𝐶 ]) with the concentration parameters [𝛼 1 , . . . , 𝛼 𝐶 ] all set to 1/𝐶. Then the generated styles 𝜇 𝑔𝑠𝑒𝑒 and 𝜎 𝑔𝑠𝑒𝑒 can be expressed as follows:EQUATIONEQUATIONwhere 𝜇 𝑏𝑎𝑠𝑖𝑠 and 𝜎 𝑏𝑎𝑠𝑖𝑠 are the 𝐶 global style elements (basis styles). Discussion. For the sampling method in GSEE, our objective is to obtain global style elements as basis 𝑆 from the style space 𝑉 𝑠 of the source domain. Thus, we can employ various sampling methods, such as K-means and Farthest Point Sampling (FPS) [42] . In Section 4.4, we compare and analyze the efficacy of various sampling methods through quantitative experiments and select the FPS method as the optimal approach. For sampling weights method, we can see from Equations 3 and 4, the sampling weights are utilized to obtain 𝜇 𝑔𝑠𝑒𝑒 and 𝜎 𝑔𝑠𝑒𝑒 through the global style elements. So, it is imperative to ensure that the sampling weights are non-negative and normalized. Non-negativity avoids invalid real style inversion, while normalization balances the contribution of each element, ensuring numerical stability representing style statistical characteristics. A commonly employed distribution that satisfies these two properties is the Dirichlet distribution. In Section 4.4, we validate the rationality and effectiveness of employing this distribution through quantitative experiments.Method. Utilizing only the 𝜇 𝑔𝑠𝑒𝑒 and 𝜎 𝑔𝑠𝑒𝑒 obtained from the GSEE may fail to address out-of-distribution scenarios effectively. This is attributable to two reasons: i) styles generated by the GSEE module are likely to be more densely concentrated near the source domain styles; ii) the target domains are entirely unseen, and the source domain's style space is just a small subspace within the totality of style spaces. Fig. 4 validates the existence of these two issues.A LUP module is thus proposed to complement GSEE. This LUP module aims to represent potential domain shifts by introducing randomness to generate new mean and variance statistics for feature representation. In the LUP, we posit that the distribution of the mean and variance feature statistics adheres to a standard Gaussian distribution. Then, we can model the uncertainty of domain differences through a standard Gaussian distribution 𝜇 𝑢𝑛 , 𝜎 𝑢𝑛 ∼ N (0, 1). However, due to the disparate scales of the mean and standard deviation of feature samples obtained in each batch, directly employing the sampled uncertainties 𝜇 𝑢𝑛 and 𝜎 𝑢𝑛 may result in the network generating samples that are not realistic. Similarly to [28] , in order to scale the sampled uncertainties to the same level as the real feature statistics in the source domain's features, we normalize 𝜇 𝑢𝑛 and 𝜎 𝑢𝑛 using the standard deviation among the source domain feature samples within the same batch. So, the generated styles uncertainties 𝜇 𝑙𝑢𝑝 and 𝜎 𝑙𝑢𝑝 can be expressed as follows:EQUATIONEQUATIONwhere 𝜎 𝑏 ( * ) represents the standard deviation across batch dimension, • denotes the Hadamard product.Constraint. As discussed in the part of method, due to the invisibility of the target domain, we simulate out-of-distribution scenarios by normalizing a standard Gaussian distribution with information from the source domain. However, this strong assumption of a Gaussian distribution under weak constraints (normalization) may result in the network producing an excessive number of unrealistic style images, thereby negatively impacting network performance. Therefore, we additionally propose a Density Distribution Consistency (DDC) loss L 𝑑𝑑𝑐 to constrain this issue.Given the need to consider both the diversity and realism of data styles, we do not directly impose our DDC loss function on the discrepancy between generated and source real styles. Instead, we constrain the outcomes obtained from different stylized images. This is because if a style falls outside the distribution range, the resulting error could be significant. We utilize the Jensen-Shannon Divergence as a loss function to assess the posterior probability between the generated styles and the source styles results as follows:EQUATIONwhere KL means Kullback-Leibler Divergence, 𝐴 = 1 2 (𝑝 (𝜓 (𝜑 (𝑥 𝑠 )))+ 𝑝 (𝜓 ( x𝑠 ))), x𝑠 means the generated new stylized samples and 𝜓 is the decoder.Data Augmentation. We observe that in crowd counting, the stylistic disparities between domains often manifest in the form of weather and illumination variations across different scenes. Compared to the domain shift between real-world and synthetic scenes, these shifts are relatively minor. To ensure the realism of the styles generated by the UGSDA method, we blend the feature statistics representative of the real data's styles from each batch with the newly generated styles. In Section 4.4, we further demonstrate the efficacy of the blending operation. To ensure computational simplicity, we obtain the final style by simply averaging as follows:EQUATION𝜎 (𝑥 𝑠 ) = 𝑚𝑒𝑎𝑛(𝜎 𝑔𝑠𝑒𝑒 (𝑥 𝑠 ) + 𝜎 𝑙𝑢𝑝 (𝑥 𝑠 ), 𝜎 (𝑥 𝑠 ))By replacing the style representation 𝜇 (𝑦) and 𝜎 (𝑦) in Eq. 1 with newly generated styles 𝜇 (𝑥 𝑠 ) and 𝜎 (𝑥 𝑠 ), we can generate stylized samples f 𝑠 aligned with the content of the input from the source domain. Loss Function. The loss function for the training of the proposed method consists of two portions. One is the Mean Square Error loss L 𝑚𝑠𝑒 , which is widely used to evaluate the difference between the estimated density maps and the ground truth density maps. The formula is as follows:EQUATIONwhere 𝑑 𝑖 is the predicted density map, Gauss(•) is the Gaussian operation and 𝑔 𝑖 is the annotated dot map. This becomes 2𝐵 because data volume is doubled each training iteration through data augmentation.Another is the density distribution consistency loss L 𝑑𝑑𝑐 which is introduced in Section 3.3. Finally, the total loss function of our proposed method is defined as:EQUATION)where 𝜆 is a hyperparameter, and we set to 0.1. Network Pipline. During the training phase, input images are initially processed through an encoder network for feature extraction, yielding source domain image features 𝑓 𝑠 . These features, are subsequently processed by both the GSEE and LUF modules, producing two sets of means and variances. Following this, the final styles are sampled utilizing the data augmentation methods depicted in Equations 8 and 9. Finally, stylized samples f 𝑠 are obtained using Equation 1. Both the 𝑓 𝑠 and f 𝑠 are fed into the decoder and the entire network is trained under the constraint of L.During the testing phase, all target domain images are directly processed through the encoder and decoder components to obtain the final results. The proposed UGSDA method does not participate in the testing phase, thereby not affecting the inference speed. 
experiments 4.1 datasets: 
 To evaluate the effectiveness of our method, we conduct domain generalization experiments on four representative datasets including Shanghai Tech A/B [62] , UCF_QNRF [20] and NWPU [49] . Shanghai Tech A/B. The ShanghaiTech dataset comprises two subsets, ShanghaiTech Part A (SHA) and B (SHB). SHA has a total of 482 images of varying resolutions; each image has 501 individuals on average, making it a notably congested dataset. The resolution of SHB is 768 × 1024, and each image contains only 123 individuals on average, making it comparatively sparser than the SHA dataset. UCF_QNRF. This dataset contains 1,535 images with a total of 1,251,642 head annotations. In this dataset, 1201 images are the training set, and the remaining 334 images are the test set. NWPU. NWPU is a large-scale crowd counting dataset comprising 5,109 images annotated with 2.13 million labeled points. In this dataset, 3,109 images are split for training, 500 images comprise the validation set, and the remaining 1,500 images, with hidden annotations, serve as the test set. We obtain the performance by submitting the counting results on the test set to the NWPU evaluation system.Implementation Details. The same backbone as DGCC [8] is used to compare our method with the earlier algorithm fairly. For density estimation, we employ two convolutions to obtain feature maps with the same number of channels as DGCC, and then use a structure identical to DGCC's to predict the density map. The Adam [24] algorithm is used to optimize the network. Hyperparameters 𝜆 and 𝑘 are set to 0.1 and 3. All experiments are conducted on Nvidia 3090 GPUs and the batch size is set to 4. The ground truth density maps for SHA, SHB and UCF_QNRF are generated using code from C-3 framework [11] within MATLAB. Following the previous work [49] , we produce corresponding ground truth density maps for NWPU. Evaluation Metrics. To quantitatively assess the performance of each model, we adopt two widely used metrics in crowd counting: Mean Absolute Error (MAE) and Mean Squared Error (MSE).Quantitative comparison. In Table 1 , we conduct experiments with SHA and SHB as the source domains, and SHB and UCF_QNRF or SHA and UCF_QNRF as the target domains. We categorize the experimental results into three paradigms: Domain Adaptation (DA), Fully-supervised on source domain(NoAdapt), and Domain Generalization (DG), with only DA undergoing finetuning on the target domain images. Compared to the DA methods requiring target domain images in the first part of Table 1 , our approach achieves comparable performance. Notably, when SHB serves as the source domain, our method reaches the best performance without the need for finetuning on target domains. As shown in the second part of Table 1 , compared to fully supervised on source domain crowd counting methods, our method achieves the best results across all four domain generalization settings. These results demonstrate that while crowd counting methods may perform well in the source domain, they are not universally applicable across different domains.Compare with DG-MAN, which is a domain generalization result obtained from DGCC through the integration of a crowd counting method MAN [32] and a domain generalization method Agr-Sum [40] . It is apparent that although their performance improved in new target domains, there remains a significant gap compared to our method. This suggests that simply incorporating domain generalization techniques into crowd counting methods is inadequate; tailored design to the crowd counting methods is necessary.Compared with a tailor-designed DGCC, which uses meta-learning technique for crowd counting domain generalization, our method achieved the best results in all four target domains. A potential reason is that their method does not essentially increase data diversity or assign new tasks to the network. Instead, it splits the source domain into meta-training and meta-test subsets to bolster the network's ability to extract domain-invariant features across domains. In contrast, our method enhances the network's generalization ability to unknown domains by employing data augmentation techniques to expose the network to a broader range of new style images. Furthermore, our approach does not affect the network's inference speed, while DGCC requires an additional DICM module during network inference, which slows down the inference time.Following DGCC, we also conduct experiments with the same model parameters as detailed in Table 2 . All of these results further validate the robust generalization capability of our method in the unseen target domains. Qualitative comparison. Fig. 3 presents a qualitative comparison between DGCC and our method when generalized to the UCF_QNRF dataset. Even in styles that are almost absent from the training set, such as grayscale images, dimly lit scenes, and colorful lighting, our method can predict a high-quality density map that accurately reflects the crowd density and the total number of people in the image. In DGCC, because the network is limited to learning generalizability solely from existing data, their performance suffers in these scenarios, particularly in distant and densely crowded regions.In this section, we validate the robustness of our method across various models, while also discussing the effectiveness of each component and the principles behind the selection of hyperparameters. Furthermore, we analyze the choice of methods such as FPS [42] and KL divergence, validating their rationality and effectiveness. Robustness to different networks. In order to prove the effectiveness of our proposed method, we apply it to two other crowd counting networks, CSRNet [29] and SFCN [50] . Taking SFCN as an example, when the model trained on the SHA dataset is directly tested on UCF_QNRF, the MAE and MSE metrics are 174.3 and 294.0, respectively. After incorporating our proposed UGSDA method, the model shows a significant improvement of 47.1 (27.02%) and 74.5 (25.34%) in the MAE and MSE metrics, without finetuning. As shown in Table 3 , when employing our method into these networks, there is a varying degree of performance improvement in terms of MAE and MSE. These results prove that our method not only possesses domain generalization capabilities but also exhibits universality across different networks. Ablation of GSEE. We first investigate the impact of the insertion position of GSEE on network performance. GSEE_𝑖 denotes the insertion of GSEE after the 𝑖-th max-pooling layer, where GSEE_1 is the strategy in Ours. As shown in Table 4 , the network performance declines when we apply the GSEE module at deeper layers of the network's output. This is because in CNNs, shallow layers typically capture more style-related information, while deeper layers yield more semantic information [19, 59] . Since GSEE needs to capture the global style elements from the source domain features, the performance is optimal in our setup using GSEE_1. We also give some ablation studies about the sampling method. As shown in Table 4 , using k-means for sampling global style elements results in a certain degree of performance degradation than Ours (FPS). This is because the k-means algorithm's insufficient focus on margin styles in the source domain, leading to a more significant performance drop on the UCF_QNRF dataset, which has a larger domain gap with the source domain. For the combination weights, we give the results compare to Gaussian Distribution (GD) which is a common used distribution. As we discussed in Section 3.2, As shown in Table 5 , a performance decline was observed when only LUP was added, compared to +GSEE. However, this result is not surprising. As analyzed in Section 3.3, this is attributed to the introduction of a hypothetical perturbation by LUP under unconstrained conditions, which is likely to negatively impact performance. Therefore, to impose constraints and balance the diversity and realism of generated styles, we further proposed the DDC loss and the Blend method. It is observable that performance improved upon the individual addition of DDC loss and the Blend method, and the more potent constraining ability of Blend made our method superior to +GSEE. When both of them are incorporated, the model achieves optimal performance. To further validate the effectiveness of the KL divergence within the DDC loss in rendering our stylized images more realistic by measuring distribution differences, we employ Cosine Similarity (CS) as a substitute to measure the distribution differences. As demonstrated in Table 5 under DDC 𝑤 CS, employing KL divergence facilitates a more accurate measurement of distribution differences, thereby yielding enhanced performance. As demonstrated by LUP 𝑤/𝑜 Norm in Table 5 , the absence of scale constraints leads to a decline in performance, underscoring the significance of normalization. Intuitively, normalizing uncertainty within LUP using the standard deviation across the entire dataset would be more effective. However, extracting features and updating for the entire dataset in each iteration is computationally unacceptable. Therefore, we use the standard deviation of each batch for normalization to optimize efficiency and accuracy. Qualitative ablation of LUP. We further substantiate the rationale behind incorporating LUP through visual analysis. Fig. 4 presents the visualization of styles obtained using GSEE module and UGSDA method. It is observed that if only the GSEE module is used, the style distribution resembles that of the source domain. However, upon incorporating the LUP module, the distribution tends to disperse, encompassing a broader range of diverse styles. Given the invisibility of the target domains, this diversification is crucial for enhancing the network's generalization ability. Different data augmentation methods. We also compare our method with various data augmentation methods. We first conduct data augmentation using the "random" setting from MixStyle [66] . Since MixStyle only considers source styles, its performance is inferior to ours. After that, we follow RobustNet [5] for photometric transformations. As a manually predefined data augmentation method, it struggles to encompass all possible styles. Hence, this approach underperforms on the UCF_QNRF dataset, which exhibits more significant domain shifts. Compared to these methods, our proposed method is validated to generate more diverse and realistic samples by comprehensively considering both the existing real styles in the source domain and potential unknown styles, thereby enhancing the model's generalization capability. Ablation of hyperparameters. We determined the parameter 𝜆 for the DDC loss through a series of ablation experiments. As depicted in the left graph of Fig. 5 , a line chart illustrates the network performance variations with changes in 𝜆, using SHA as the source domain and QNRF as the target domain. The optimal value of 0.1 was selected as the final parameter choice.We then study the impact of the basis styles 𝜇 𝑏𝑎𝑠𝑖𝑠 and 𝜎 𝑏𝑎𝑠𝑖𝑠 on network performance under varying update frequencies 𝑘. As shown in the right graph of Fig. 5 , the model achieves the best performance with an update frequency 𝑘 = 3. This is because when the update frequency is too rapid, the model becomes overly focused on the source domain, losing diversity. Conversely, a lower update frequency leads the model to deviate from the basis styles provided by the source domain, resulting in a loss of realism. Besides considering performance, the computational complexity is also feasible, making 𝑘 = 3 as our chosen setting. 
conclusion and future work: 
 In this paper, we pioneeringly explore style-based data augmentation in the domain generalization of crowd counting. An uncertaintyguided style diversity augmentation method is proposed. To acquire diverse and real samples, the method employs a global style element extraction module to sample global style elements from the entire source domain, complemented by local uncertainty perturbations at the batch level for uncertainty styles estimation. Besides these, we introduce the density distribution consistency loss and blend to further constrain the density distributions of real and stylized samples, enhancing the realism of the stylized samples. Comprehensive experiments validate that our method effectively enhances the network's domain generalization capabilities across various datasets and network architectures.In future work, we aim to investigate the domain generalization from synthetic to real datasets, a task that presents greater challenges due to the substantial domain gap and more pronounced style disparities between them.  
source domain: 
 planning across various domains. In the past decade, with the rapid advancement of artificial intelligence, crowd counting has also made remarkable progress [21, 31, 33, 35, 39, 44, 62, 65] . Almost all of these methods assume that the training and test sets are drawn i.i.d. from the same underlying distribution. However, this assumption is often violated in real-world scenarios due to substantial variations in data acquisition devices, data collection environments, and other factors. Such domain shifts between the source and target domains can lead to significant performance degradation.To alleviate this issue, several studies attempt to incorporate Domain Adaptation (DA) techniques within the realm of crowd counting [7, 16, 37, 50, 52, 53, 69] . These methods aim to minimize the domain gap by learning domain-invariant feature representations or style information specific to the target domain. While these methods yield promising results, their efficacy depends on access to images in the target domain. However, obtaining sufficient data from target domains may encounter numerous challenges, especially as people become increasingly aware of privacy protection. What is more, it is troublesome and time-consuming to finetune the pre-trained model for each target domain.As shown in Fig. 1 , compared to fully-supervised on source domain methods that perform worse on target domain, and DA methods that necessitate target domain images, Domain Generalization (DG) techniques can achieve commendable performance without any access to the target domain. Meta-learning and data augmentation are two important categories in the domain generalization. MLDG [26] proposes to use meta-train and meta-test sets to simulate the domain shifts to learn domain-invariant features. Building on MLDG, DGCC [8] employs meta-learning technique for domain generalization in crowd counting. However, DGCC suffers from a complex network structure and sensitive hyperparameters, and it requires maintaining an additional domain-invariant crowd memory which adversely impacts inference speed. AdaIN [19] is a data augmentation technique that can transmute images into any arbitrary style while preserving content. This capability enables the network to learn from a more diversified set of data within the source domain and consequently achieve a more general model. Leveraging AdaIN, we first propose to use data augmentation technique in crowd counting, which can bolster the generalization ability of models without impacting the network architecture and inference speed.In this paper, we propose an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method for domain-agnostic crowd counting. The proposed UGSDA method comprises a Global Style Element Extraction (GSEE) module, a Local Uncertainty Perturbation (LUP) module and the Density Distribution Consistency (DDC) loss. Inspired by [63] , the GSEE module represents the 𝐶-dimensional style space by sampling 𝐶 global style elements. To address the potential limitations of source domain data in capturing all possible styles, we employ the LUP during the generation process of each training batch to introduce uncertainty into the global style elements, thereby enabling the creation of a diverse set of styles. LUP calculates the mean and variance at the batch level for the images, and samples from a Gaussian distribution to obtain perturbation values for the global style elements. To prevent the stylized samples from deviating too much from real-world scenarios, we also employ the proposed DDC loss to regularize the difference between the density distribution of real samples and stylized samples in the highdimensional space. These components together generate sufficiently diverse samples during the training process, thereby enhancing the model's generalization capabilities. The contributions of this paper are summarized as follows:• To our best knowledge, this is the first attempt to employ data augmentation techniques for domain generalization in the field of crowd counting, offering advantages in terms of simply and plug-and-play functionality without affecting inference speed.  
 
 Appendix: The results on NWPU are obtained through submitting to https://www. crowdbenchmark.com/This network is modified as describe in Section 4.2. 


Decision: This submission presents the Uncertainty-Guided Style Diversity Augmentation (UGSDA) method to improve crowd counting algorithms' performance in unseen domains by generating diverse and realistic training samples solely from the source domain. The method, incorporating Global Styling Elements Extraction (GSEE), Local Uncertainty Perturbations (LUP), and Density Distribution Consistency (DDC) loss, demonstrates good generalization capabilities across various datasets and models.

All reviewers acknowledge that the proposed method is simple, clear, and achieves good performance. The initial concerns about unclear visualization, missing baseline, and clarity of component analysis were addressed in the rebuttal. Please ensure the response is included in the revised version, and make the code publicly available as promised.

Reviews: review: paper_summary: paper_summary: The paper addresses the challenge of domain shift in crowd counting algorithms, which struggle to perform well in unseen domains. Traditional domain adaptation methods require images from the target domain and additional training time, making them impractical when target domain images are hard to acquire. The authors propose an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method to overcome this limitation. UGSDA enables models to train solely on source domain images and generalize to unseen target domains by generating diverse and realistic samples during training.
strength_weakness: strength_weakness: This paper presents an approach in the field of crowd counting by employing data augmentation techniques for domain generalization. The method offers a simple, plug-and-play solution that does not impact inference speed. The proposed Uncertainty-Guided Style Diversity Augmentation (UGSDA) method dynamically generates diverse training samples by combining global style elements from the source domain with slight perturbations from each training batch. Furthermore, the Density Distribution Consistency (DDC) loss effectively ensures the realism of the density distribution in the stylized samples. This innovative approach enhances the model's generalization capabilities without the need for additional training time or target domain images.
limitations: limitations: In Table 1, the performance gain seems to be marginal on SHB with SHA as source domain.
paper_summary: paper_summary: The paper introduces an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method to improve domain generalization in crowd counting by generating diverse training samples. UGSDA incorporates Global Styling Elements Extraction (GSEE), Local Uncertainty Perturbations (LUP), and Density Distribution Consistency (DDC) loss. While the approach shows promise, the paper lacks sufficient comparison with existing methods, clear visualization of style differences, and proof of its innovative aspects. Additionally, some figures are unclear.
strength_weakness: strength_weakness: 1. The paper is well-structured, and the methods are described in a clear and understandable manner.
2. The paper includes a comprehensive set of experiments.
3. The experimental results are competitive and demonstrate the effectiveness of the proposed method.
limitations: limitations: 1. Inference Speed Comparison: The inference speed of DGCC is not comparable to demonstrate advantages.
2. Style Unclear: Style is noticeable in many tasks but unclear in crowd counting. The paper does not visualize the style differences in different images or provide specific explanations of the styles.
3. Impact of Style Differences on Generalization: The paper needs to prove how style differences affect domain generalization performance.
4. Diversity of Source Domain Samples: The diversity of source domain samples limits the method's generalization ability.
5. Lack of Innovation: The method appears to be a transfer of techniques from other fields to the counting domain. The paper needs to compare this work with reference [63] to highlight its unique contributions.
6. Clarity of Figure 2: Figure 2 is unclear, and the FPS is not explained. Additionally, there is no visualization of style transformation before and after augmentation, making it hard to understand.
paper_summary: paper_summary: This paper proposes an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method, enabling the crowd-counting models to be trained solely on the source domain and directly generalized to different unseen target domains. UGSDA generates diverse and realistic samples with a GSEE module, LUP module, and DDC loss where GSEE extracts global style elements, LUP obtains uncertainty perturbations, and DDC loss regulates the extent of perturbations.
strength_weakness: strength_weakness: - The first attempt is to employ data augmentation techniques for DG in crowd counting.
- Plug-and-play functionality without affecting inference speed.
- Extensive experiments with benchmarks on different scales demonstrate the effectiveness of the proposed method.
limitations: limitations: - **Insufficient Baselines**: It would be good to compare the proposed method with existing DG data augmentation methods such as CrossGrad, DDAIG, and MixStyle. Although these methods are always applied to image datasets, they can also be applied to different data types.
paper_summary: paper_summary: The paper introduces an Uncertainty-Guided Style Diversity Augmentation (UGSDA) method to overcome the performance barriers posed by domain shift in crowd counting algorithms. As acquiring target domain images can be challenging, UGSDA enables crowd counting models to be trained exclusively on the source domain and generalize effectively to unseen target domains. This is achieved through the integration of three key components: Global Styling Elements Extraction, Local Uncertainty Perturbations, and Density Distribution Consistency loss. Extensive experiments validate the approach's strong generalization capabilities across various datasets and models.
strength_weakness: strength_weakness: 1) The paper proposes a novel Uncertainty-Guided Style Diversity Augmentation (UGSDA) method that effectively addresses the domain shift challenge in crowd counting.
2) The integration of global styling elements extraction, local uncertainty perturbations, and density distribution consistency loss creates a robust framework that generates realistic and diverse samples during training.
3) The paper provides comprehensive experiments across various datasets and models, demonstrating the strong generalization capabilities of the UGSDA method. This adequate evaluation validates its effectiveness in addressing domain shift.
limitations: limitations: 1) The paper mentions a complex computing evaluation compared to DGCC but does not elaborate on it. Clarification may be needed.
2) Fine-grained framework and visualizations analytics of the GSEE and LUP components are missing, which hampers understanding of the method.