Weight Diffusion for Future: Learn to Generalize in Non-Stationary Environments

Authors: mixue  xie
shuang  li
binhui  xie
chi harold liu
jian  liang
zixun  sun
ke  feng
chengwei  zhu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 10:43:45

Abstract: Enabling deep models to generalize in non-stationary environments is vital for real-world machine learning, as data distributions are often found to continually change. Recently, evolving domain generalization (EDG) has emerged to tackle the domain generalization in a time-varying system, where the domain gradually evolves over time in an underlying continuous structure. Nevertheless, it typically assumes multiple source domains simultaneously ready. It still remains an open problem to address EDG in the domain-incremental setting, where source domains are non-static and arrive sequentially to mimic the evolution of training domains. To this end, we propose Weight Diffusion (W-Diff), a novel framework that utilizes the conditional diffusion model in the parameter space to learn the evolving pattern of classifiers during the domain-incremental training process. Specifically, the diffusion model is conditioned on the classifier weights of different historical domain (regarded as a reference point) and the prototypes of current domain, to learn the evolution from the reference point to the classifier weights of current domain (regarded as the anchor point). In addition, a domain-shared feature encoder is learned by enforcing prediction consistency among multiple classifiers, so as to mitigate the overfitting problem and restrict the evolving pattern to be reflected in the classifier as much as possible. During inference, we adopt the ensemble manner based on a great number of target domain-customized classifiers, which are cheaply obtained via the conditional diffusion model, for robust prediction. Comprehensive experiments on both synthetic and real-world datasets show the superior generalization performance of W-Diff on unseen domains in the future.

Main paper: 
 introduction: 
 Domain generalization (DG) deals with a fundamental problem in modern machine learning [13, 46, 47] , where performance degeneration often occurs when deep models encounter out-of-distribution (OOD) data [38, 56] . The goal of DG is to learn a model that can perform well on unseen target domains by leveraging labeled data from multiple related but different source domains [23, 19, 57, 22] . Despite of abundant works on DG and promising progress so far, they typically embark upon the generalization among stationary and discrete environments [29] , where the distribution shift among domains is obvious and remains static over time. In contrast, there have emerged several works on evolving domain generalization (EDG) [24, 29, 51, 2, 44, 50] in recent years, where the data distribution gradually shifts in an underlying continuous structure, e.g., the age-related structural changes in the optic nerve in ocular diseases [12] . But most of EDG methods still assume multiple source domains simultaneously ready, which may be impractical in real world. As the data distribution constantly evolves along time, training data from new data distributions will continue to emerge. Hence, equipping models with lifelong learning ability is crucial for their practical applications.Despite of the fact that existing researches on continual learning [16, 32, 5] have studied the empowerment of lifelong learning, their focus is on maintaining the performance of seen tasks, instead of generalizing on unseen domains in the future. Therefore, it is still an open problem to achieve evolving domain generalization in the domain-incremental setting, where source domains sequentially arrive to mimic the dynamics of training domains. To this end, previous work EvoS [45] models the features from each domain as a Gaussian distribution and proposes to capture the evolving pattern at the feature level by leveraging self-attention mechanism to generate the feature mean and variance for future domain based on those of historical domains. However, the assumption that features adhere to a Gaussian distribution may not always be applicable. Different from EvoS, we propose to excavate evolving pattern at the parameter level and further achieve domain-customized parameter generation.Inspired by neural network diffusion [42] that there exist specific parameter patterns in optimized model layers and these patterns can be modeled with diffusion model, we propose to capitalize on the strong modeling ability of diffusion models to capture the evolving pattern of optimized classifiers across domains. To achieve this, we propose a Weight Diffusion (W-Diff) approach, which is specialized for EDG in the domain-incremental setting. Specifically, to address the problem of inaccessible historical data, we maintain a first-in-first-out (FIFO) queue to store the optimized classifier weights of historical domains. The stored classifier weights of each historical domain serve as a reference point to calculate the change between the classifier weights of current domain (regarded as the anchor point) and that of corresponding historical domain. The changes of classifier weights between the anchor point and different reference points provide evolving patterns at different time intervals, which can be utilized to make the modeling of evolving patterns more robust.In addition, for guidance on how to switch from a reference point to the anchor point, we condition the diffusion model on the class prototypes of current domain along with the reference point. Then, the conditional diffusion model is trained to model the distribution of residual classifier weights, i.e., the change of classifier weights between the anchor point and reference point. Meanwhile, to reduce the overfitting of the feature encoder and to restrict the evolutionary pattern to be reflected only in the classifier as much as possible, we learn a domain-shared feature space by enforcing the predictions from different classifiers to be consistent. Finally, during the inference stage, we adopt weights ensemble to give robust predictions based on a great number of generated classifier weights by the diffusion model that is conditioned on current class prototypes and different reference points.Contributions: 1) We study the under-explored area of evolving domain generalization in the domain-incremental setting and explore the innovative usage of diffusion model for this problem.2) We propose a novel weight diffusion (W-Diff) approach to capture the evolutionary pattern at the parameter level, orthogonal to previous feature level approaches. Capitalizing on the strong generative ability of diffusion model, W-Diff can generate customized parameters by controlling the condition and make robust predictions via weights ensemble. 3) Comprehensive experiments on both synthetic and real-world datasets verify the effectiveness and superiority of W-Diff on generalization. 
related work: 
 Evolving Domain Generalization (EDG) learns the evolving pattern underlying in multiple source domains to achieve generalization capability on the unseen future target domains over time [2, 50, 24, 52, 44, 29] . To name a few, SDE-EDG [50] introduces stochastic differential equations (SDEs) to model the evolving pattern through individual temporal trajectories. DRAIN [2] builds on the Bayesian framework and leverages LSTM to infer the future status of the whole network. However, most of these methods, except for DRAIN, require multiple source domains to be simultaneously available. Very recently, EvoS [45] focuses on EDG with sequentially arriving domains, considering the low efficiency of training the model from scratch once the accumulated domains are updated. It assumes that features for each domain follow a Gaussian distribution and then models the evolving pattern of the feature distribution, while this assumption is not always suitable. Besides, in the generalization process of multiple consecutive target domains, the statistics generated from previous timestamps are used as inputs to the attention mechanism to generate the statistics at next timestamp, which in turn serve as the input for next generation. This manner is likely to cause error accumulation if previous generation is not accurate. Orthogonal to EvoS [45] , we proposes to mine the evolving pattern at the model parameter level and further implement domain-oriented parameter generation via controlling the condition of the diffusion model to avoid the potential error accumulation in EvoS.Continual Learning (CL) focuses on the scenario where the model is trained on a sequence of tasks, and the model is required to adapt to current task and meanwhile maintain the performance on previous tasks [53, 16, 5, 39, 43, 32, 11] . The literature on this field is abundant. Most of the techniques can be categorized into architecture-based [34, 9] , representation-based [4, 43, 10] , regularization-based [16, 53, 32] and replay-based [5, 39, 11] . In this work, we also continually train models on sequential domains, but the goal is to generalize well on novel domains in the near future.Parameter Generation has been gaining great interests with the rise of diffusion models [42, 8, 21, 54] . For example, p-diff [42] directly generates high-performing neural network parameters from random noises with a standard latent diffusion model, which verifies the feasibility of modeling the parameter distribution via diffusion models. Nevertheless, p-diff uses unconditional diffusion model and can only generate parameters for in-distribution data. G.pt [28] collects the loss, error or return of task model checkpoints during training as the condition for the diffusion model. However, it is designed for a single dataset to which the training data belongs, thus struggling with distribution shifts. D2NWG [36] uses CLIP [30] to extract features for each sample and leverages Set Transformer [18] to generate dataset encoding from these features. Then, the diffusion model is conditioned on the dataset encoding. But labeled training set samples of a new dataset are required to obtain the dataset encoding, which is infeasible in unlabeled target domains. In addition, ProtoDiff [8] and MetaDiff [54] generate prototype classifiers for the meta-test stage by conditioning the diffusion model on the information (e.g., the prototypes in [8] and the gradients in [54] ) from the support set. Yet, they concentrate on few-shot learning and the support set requires labeled data, which is unavailable in the target domain of EDG. By contrast, we aims at capturing the evolving pattern among source domains and leveraging it to enable generalization on future target domains without any labeled target data. 
problem formulation: 
 We consider the evolving domain generalization (EDG) in the domain-incremental setting. Formally, during training phase, we are given T sequentially arriving source (training) domains: S = {D 1 , D 2 , . . . , D T }, which are collected at timestamps t 1 < t 2 < . . . < t T , respectively. Each domain is defined as D t = {x t i , y t i } N t i=1 , t = 1, . . . , T , where x t i is the i-th sample from the t-th domain, y t i ∈ {0, 1, . . . , C -1} is the category label of sample x t i , and N t , C are the number of training samples in the t-th domain and the number of categories, respectively. In the domainincremental setting, we can only access current domain D t at timestamp t t and historical domains {D 1 , . . . , D t-1 } are unavailable. This takes into account the data storage burden, privacy concerns and the dynamic evolution of the source domain. Following previous EDG works [24, 29, 2] , the label set is the same among domains, but the data distribution of domains is assumed to continuously evolve over time in some patterns. And our goal is to generalize the model, which is composed of a feature encoder E ψ parameterized with ψ and a classifier H W parameterized with W, on unseen K target (testing) domains in the future: T = {D T +1 , . . . , D T +K }. To tackle this problem, we propose to model the evolving pattern at the parameter level via the conditional diffusion model and generate customized parameters for future domains by controlling the condition of the diffusion model.Diffusion models have achieved tremendous success in computer vision by modeling the probability transformation from a prior Gaussian distribution to the target distribution [14, 40] . They typically comprise a diffusion process to progressively add Gaussian noise to data in a multi-step Markov chain and a denoising process to recover data from the noise via reversing the diffusion process.Diffusion process. Given a clean data point x 0 sampled from a real data distribution q(x), i.e., x 0 ∼ q(x), the diffusion process is characterized as a Markov chain which slowly adds random Gaussian noise to x 0 in S steps, obtaining a sequence of noisy samples: x 1 , . . . , x S . Formally, this process is expressed asEQUATIONwhere {β s ∈ (0, 1)} S s=1 is a variance schedule, N represents Gaussian distribution, and I is the identity matrix. And the forward diffused sample at step s, denoted as x s , can be directly obtained in a single step by Eq. ( 2) without iteratively adding noise:EQUATIONwhere ᾱs = s s =1 (1 -β s ). When step size S approaches infinity, x S is equivalent to a data point from an isotropic Gaussian distribution, i.e., the prior Gaussian distribution N (0, I).Denoising process. Given a start noise x S ∼ N (0, I), the denoising process moves backward on the multi-step Markov chain as s decreases from S to 1 to remove the noise at each step s, finally recovering the clean data. Concretely, the formulation of the denoising process at step s is denoted asEQUATIONwhereµ θ (x s , s) = 1 √ 1-βs x s -βs √ 1-ᾱs E θ (x s , s) and E θ (•, •) is a denoising model parameterized with θ to estimate the noise. σ s is a variance hyperparameter that is theoretically set to σ 2 s = β s in most diffusion works [14, 26] . During the training stage, the denoising model E θ is trained by minimizing the following loss L dif f to minimize the noise estimation error:EQUATION)Conditional diffusion model. The way of conditional diffusion models to generate samples is analogous to the unconditional one, except for the added conditional information. Specifically, as in most conditional diffusion works [25, 17] , the denoising model E θ (x s , s) is replaced with E θ (x s , s, c), where c denotes the condition, e.g., class labels, texts, images, etc. The matched condition c regulates the sample generation in a supervised manner to ensure the desired image content. And inspired by the generating of specific image contents via introducing conditional information to diffusion models, we propose to achieve domain-oriented parameter generation by controlling the diffusion condition. 
methodology: 
 With the preliminary knowledge of EDG in the domain-incremental setting and diffusion models, we will present the details of Weight Diffusion (W-Diff) in this section. We begin with how to obtain the data for diffusion model training in Section 4.1 and then model the evolving pattern of parameters via the conditional diffusion model in Section 4.2. Finally, the inference procedure of W-Diff to generate customized classifiers is presented in Section 4.3. The overview of W-Diff is illustrated in Fig. 1 .In our approach, we try to capture the evolving pattern in optimized model parameters across domains and further generate customized parameters for target domain via leveraging the learned pattern.Considering the unaffordable training cost if modeling the whole parameters for relatively large models, we choose to excavate the evolutionary pattern in the task-specific head, e,g., the classifier for classification tasks. Nevertheless, the remaining parts of the task model would overfit to current domain and cause degraded generalization if without any processing. To avoid this, we learn a domain-shared feature encoder for all domains and a domain-specific classifier for each domain during the domain-incremental training. As t increases from 1 to T , once the training stage on historical domains, and the anchor point queue Q a and prototype queue Q p store the updated classifier weights and prototype matrix at each iteration after the warm-up stage on current domain. L t con is the prediction consistency loss to learn a domain-shared feature space and L t ce is the cross-entropy loss. The conditional diffusion model E θ is trained with the noise estimation error loss L t dif f to model the evolving pattern of classifiers, conditioned on historical reference point and current prototype matrix. domain D t ends, the classifier weights with the best validation performance on the validation set of D t , denoted as Ẅt ∈ R C×d f , is stored in the reference point queue Q r , where d f the dimension of deep features output by the feature encoder E ψ . Q r is a global FIFO queue with the maximum length L and is used to calculate the change of classifier weights between current domain and a given historical domain in Section 4.2, which reflects the evolving pattern at the parameter level.Learning Domain-Shared Feature Encoder. In the domain-incremental setting, we can only access the data from current domain, which prohibits us from utilizing conventional DG methods that require to access multiple domains simultaneously to learn domain-invariant feature representations. To tackle this problem, we resort to the stored different classifiers in Q r . Intuitively, if domain-shared feature representation is learned, classifiers from different domains should give similar predictions for a given data sample. Hence, at timestamp t t , we train the task model on the t-th domain D t by minimizing the consistency loss L t con to learn a domain-shared feature space:EQUATIONwheref t i = E ψ (x t i ) ∈ R d f , | • |is the length of the object and sg(•) denotes stopping gradients. Ẅt is the stored classifier weights of historical domain D t and W t is current classifier weights on D t .Learning Domain-Specific Classifier. As t increases from 1 to T , domain-specific classifier is directly learned by incrementally training the task model via the supervision loss L t ce on domain D t :EQUATIONOverall, when training on domain D t , the task model is optimized by the following total loss L t total :EQUATIONwhere λ is a tradeoff hyperparameter to balance the two losses.Collecting Data for Diffusion Model Training. Considering that parameters at the early stage are unstable, we start to collect the training data for diffusion model after the warm-up stage of the task model is over on corresponding domain. Specifically, when training on the t-th domain, the warm-up epochs of the task model account for ρ ∈ (0, 1) of the total training epochs on domain D t , where ρ is a hyperparameter. After the warm-up stage, the updated classifier weights W t ∈ R C×d f via back-propagating the gradients of L t total and the updated prototype matrix µ t ∈ R C×d f via Eq. ( 8) are respectively stored into the anchor point queue Q a and prototype queue Q p at each iteration.EQUATIONµ t [c] is the c-th row of µ t , i.e., the prototype of class c on domain D t based on all data in the seen batches after the warm-up stage on D t . B is the batch size of task model. n t counts the total number of samples in seen batches after the warm-up stage and is initialized to 0 at the start of training on each domain. p t,t i [c] is the predicted probability of x t i belonging to class c via current classifier of domain D t . Here, using predicted probability instead of ground-truth label to compute prototypes avoids the issue of missing categories in a batch and the inaccessibility of labels in testing domains.In implementation, Q a and Q p are both FIFO queue with the maximum length M , i.e., the training batch size of the diffusion model. When they are full, the training of the conditional diffusion model starts. Note that Q a and Q p are only used during the training phase and are locally used on each domain. That is, they are initialized to empty at the beginning of the training stage on each domain.Having prepared the data for diffusion model training, we can utilize them to learn the evolving pattern of parameters during the domain-incremental training process of the task model. To be specific, we use the difference between the classifier weights of current domain and that of a given historical domain to represent the evolution of parameters:EQUATIONwhere W t m , m = 1, 2, . . . , M, is the classifier weights of current domain D t , which is cached in the anchor point queue Q a when training the task model on D t . And Ẅt is the classifier weights of the historical domain D t from the reference point queue Q r , t ∈ {t -|Q r |, . . . , t -1}. Hence, the residual classifier weights {∆W t,t m } M m=1 represent how to evolve from a reference point to the anchor point. Moreover, to guide the evolution, we provide the paired condition for each residual classifier weight matrix ∆W t,t m , where the paired condition is formulated as c t,t m = Ẅt ⊕ µ t m ∈ R C×d f ×2 and ⊕ denotes concatenating. The additional condition provides the information about the the starting point and knowledge about the distribution of data in the feature space, which is rightly the anchor point, i.e., the optimized classifier, needs to adapt to. Then, when the task model is incrementally trained on the t-th domain, the conditional diffusion model is also incrementally trained to minimize the following noise estimation error loss L t dif f in Eq. ( 10), so as to learn how to generate the desired residual classifier weights when given the reference point and prototype matrix as the condition.EQUATION)In implementation, the conditional diffusion model adopts the similar U-Net structure as LDM [31] and uses a hybrid conditioning way, i.e., the condition is injected both in the cross-attention and input sides. In Eq. ( 10), different reference points could enrich the diversity of training data for the conditional diffusion model and provide the evolving pattern at different time intervals.After finishing the training on domain D T , we can use the conditional diffusion model to generate customized classifiers for a given testing domain D test . Firstly, we calculate the prototype matrix µ test of domainD test via µ test [c] = 1 N test N test i=1 ptest i [c] • f test i , where ptest i = 1 |Qr| Ẅt ∈Qr softmax( Ẅt × f test i ), c = 0, 1, . . . , C -1.Here, we use the more robust average prediction of multiple classifiers to compute the prototype matrix in the inference phase. Then, given each reference point Ẅt in Q r along with the prototype matrix µ test , we can generate M g residual classifier weights: {∆W test,t j } Mg j=1 by substituting the denoising net in Eq. ( 3) with its conditional version and applying the denoising process with condition c test,t = Ẅt ⊕ µ test . Ultimately, we use the following average weight ensemble Wtest as the final classifier for label predicting on D test :EQUATIONIn this way, we capitalize on the powerful modeling and generating ability of conditional diffusion model to cheaply produce a great number of target-customized classifiers, which offers more robust and accurate predictions. The pseudo codes of training and testing procedures are in Appendix C. 
experimental setup: 
 Benchmark Datasets. We evaluate W-Diff on both synthetic and real-world datasets [2, 48] , including two text classification datasets (Huffpost, Arxiv), three image classification datasets (Yearbook, RMNIST, fMoW) and two multivariate classification datasets (2-Moons, ONP). Except for synthetic datasets 2-Moons and RMNIST that use the rotation angle as a proxy for time, all other datasets collect real-world data with the distribution shift over time. Following [45] , the number of source and target domains is set as Yearbook: (T = 16, K = 5), RMNIST: (T = 6, K = 3), fMoW: (T = 13, K = 3), Huffpost: (T = 4, K = 3), Arxiv: (T = 9, K = 7), 2-Moons: (T = 9, K = 1), ONP: (T = 5, K = 1). For each source domain, we randomly divide the data into training and validation sets in the ratio of 9 : 1. For more details on datasets, please refer to Appendix D.1.Network Details. For the task model, we follow the usage in [48, 45] . For the conditional diffusion model, we implement it in a U-Net similar to LDM [31] . Please refer to Appendix D.2 for details.Training Details. For all datasets, we set the batch size B = 64, the loss tradeoff λ = 10 and the maximum length L = 8 for the reference point queue Q r . To optimize the task model, we adopt the Adam optimizer with momentum 0.9. As for the warm-up hyperparameter ρ, we ρ = 0.6 for Huffpost, fMoW and ρ = 0.2 for Arxiv, Yearbook, RMNIST, 2-Moons, ONP. For the conditional diffusion model, we set the maximum diffusion step S = 1000 and use the AdamW optimizer with batch size M = 32, where M is also the maximum length of queue Q a and Q p . And the number of generated residual classifier weights based on each reference point is set to M g = 32 in the inference stage. All experiments are conducted using the PyTorch packages and run on a single NVIDIA GeForce RTX 4090 GPU with 24GB memory. Three independent experiments with different random seeds are repeated for each task to report the mean and standard deviation (std) of accuracy, which is denoted in the format of "mean ± std" in the table. Please refer to Appendix D.3 for more details. Evaluation Metrics. We report the generalization performance on K target domains in the future, including the average accuracy "OOD avg." ( 1 K K k=1 Accuracy(D T +k )) and the worst accuracy "OOD worst" (min k∈{1,...,K} Accuracy(D T +k )) on K target domains and the accuracy on D T +1 .We provide the quantitative results in Table 1 , 2, 3(a), where the results of baselines in the nonincremental and incremental scenarios are reported from [45] . For ONP dataset, we notice that previous continuous domain adaptation methods (CDOT and CIDA) and EDG methods (GI and DRAIN) all perform worse than the Offline method that trains the task model on the cumulation of all source domains. In contrast, our W-Diff still obtains superior accuracy to previous state-of-the-art method (EvoS) on this challenging dataset, which validates the superiority of W-Diff. Besides, our W-Diff also achieves the best results on Huffpost, Arxiv, RMNIST and fMoW, in terms of the OOD worst accuracy. These results benefit from the modeling of parameter evolution pattern and the more robust predictions via the weight ensemble based on the conditional diffusion model. For Yearbook dataset, DRAIN, which models the evolution of whole model parameters via LSTM, is inferior to EvoS, which models the evolution of domain-level feature distribution. It suggests that modeling the evolving pattern at the feature level may be more appropriate for Yearbook, which also explains why W-Diff does not obtain the state-of-the-art performance on Yearbook. But our W-Diff still obviously outperforms DRAIN. Overall, we can observe that W-Diff surpasses the baselines in the incremental-training setup on six out of seven datasets, which shows the superiority of W-Diff.Ablation Study. Firstly, the significant performance drop of variant A in Table 3 (b) suggests that learning domain-invariant feature representations is necessary for EDG in the domain-incremental setting. Otherwise, the feature encoder could easily overfit to current domain, prohibiting the task model from generalization. Then, we try different ways to construct the average weight ensemble Wtest , including variant C which directly uses the historical classifier weights in Q r , i.e., Wtest = Decision Boundary Visualization on Future Domain. In Fig. 2 (a), the model is incrementally trained until the training stage on the t-th domain D t finishes. Then we visualize the decision boundary on the next future domain D t+1 , t = 8, 9. From the results, we can see that the decision boundary of W-Diff adapts to the evolution of domains better than that of EvoS, which shows the superiority of W-Diff in addressing evolving domain generalization in the domain-incremental setup.t-SNE Visualization of Features. In this qualitative experiment, we visualize the features of future target domains for RMNIST dataset to show the effectiveness of L t con . From Fig. 2 (b), we can observe that features from different target domains align well. To some extent, it verifies the effectiveness of L t con to learn a domain-shared feature space, which contributes to the mitigation of distribution shift. Visualization of Generated Classifier Weights. In Fig. 3 (a), we plot the generated classifier weights for domain D T +1 , T = 6, on RMNIST, as well as the accuracy range on D T +1 of generated classifier weights based on different reference points. In Fig. 3 (a), some generated weights locate close to the average fine-tuned weights W 7 , showing that W-Diff generates domain-customized classifiers. Besides, Ŵ7|1 , Ŵ7|2 , Ŵ7|3 are similar, while Ŵ7|4 , Ŵ7|5 , Ŵ7|6 are different, due to the more pronounced differences among reference points Ẅ4 , Ẅ5 , and Ẅ6 . This implies that the evolution pattern may be different at different time intervals. And these diverse and high-performing generated weights based on different reference points could conduce to more robust predictions.Evaluating in Batch-Data Stream. In addition to the inference way in Section 4.3, we also provide another version, where the data in target domain arrives batch by batch. Concretely, we use the iterative manner in Eq. ( 8) along with the average prediction in Section 4.3 to update the prototype matrix µ test , once a batch of data from D test arrives. Then, we compute the average weight ensemble Wtest via Eq. 11 for this data batch. Fig. 3(b) shows the results on RMNIST and Huffpost. The two manners present similar results and users can choose appropriate manner based on their scenarios.Equipping Condition with Timestamp Difference. In this part, we try to explicitly incorporate the timestamp difference between the anchor point and reference point into the condition of diffusion model. Concretely, we scale the reference point Ẅt in the condition c t,t by a factor η, where η = 1.5 -1 1+e -∆t and ∆t = t t -t t is the timestamp difference between reference and anchor points. More distant reference points have larger ∆t and are weakened. The results on RMNIST dataset are given in Table 4 , where the average and worst accuracies of target domains improve slightly. The insignificant performance improvement may be due to the fact that our approach implicitly takes the timestamp difference into account via the domain-incremental training and residual classifier weights.Results with Larger Backbones. We try larger backbones on the fMoW dataset by replacing the DenseNet-121 with DenseNet-169/201/161 [15] , respectively. The results are provided in Table 5 . When applying lagers backbones, our method still works well and further performance improvements are obtained. This benefits from the consideration of only modeling the evolution of classifier weights, instead of the whole network parameters. Otherwise, the training difficulty and huge memory burden from the conditional diffusion model would be unbearable, despite of the greater feature extraction capability of larger backbones. 
conclusion: 
 This work delves into the under-explored problem of evolving domain generalization in the domainincremental setting, where the source domain is also non-stationary and dynamically evolves. To tackle this, we propose a Weight Diffusion (W-Diff) approach to capture the evolving pattern across domains at the parameter level and further generate customized classifiers for future domains. W-Diff innovatively leverages the conditional diffusion model to learn the evolution of classifiers from historical domain to current domain, conditioned on the historical classifier weights and current prototype matrix. Extensive results on synthetic and real-world datasets verify the efficacy of W-Diff.  
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We have discussed the potential social impacts in Section A.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The datasets and models used in the paper are all public and from previous works.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: The datasets and models used in the paper are from previous works and are public. We have cited related papers in our work and provided the license in Section D.1.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: No new assets are released.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Neither crowdsourcing nor research with human subjects is used.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA] Justification: Our work does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: Broader Impacts. In this work, we explore the evolving domain generalization in the domain incremental setting. The ability to continually learn from dynamic source domains and leverage the learned evolving pattern to generalize on unseen domains in the future may benefit relevant nonstationary scenarios, e.g., advertisement recommendation with continually emerging new training data and autonomous driving with distribution shift over time or geographical position, etc. It reduces the time and cost for labeling data of target domain and avoids the low efficiency of training the model from scratch with all saved domains once new training domain is available. Yet, for high-security demanding scenarios, the prediction from the model should be adopted with caution to avoid severe accidents, as failures can occur in our method when facing significant distribution shifts.Limitations. Our work presents a way to capture the evolving pattern at the parameter level via capitalizing on the powerful modeling ability of conditional diffusion model. Yet, like any research, our work is not absolutely perfect. There are indeed some limitations that should be acknowledged. Firstly, the task considered in this paper limits to the classification. In the future, we may extend our method to more diverse tasks, e.g., regression tasks. Besides, considering the training cost, we only model the evolution of classifiers. Perhaps, it is feasible to consider more parameters by mapping them into a low-dimensional latent space, but achieving the accurate encoding and decoding is not easy. We leave this for a future work.Given the large number of notations used throughout the paper, we provide an overall notation description in Table 6 to ease the burden on readers. The training and testing procedures of W-Diff are presented in Algorithm 1 and 2. Randomly sample a batch of data {x t i , y t i } B i=1 from domain D t .Q r = ∅. 2 for t = 1 to T do 3 Set ψ t = ψ t-1 , W t = W t-1 , θ t = θ t-1 , Q a = ∅, Q p = ∅, µ t = 0, n t = 0.7Get the deep features of samples:{f t i = E ψ (x t i )} B i=1 .8Calculate the supervision loss L t ce in Eq. 6.L t total + = L t ce .10 if t > 1 then 11Calculate the consistency loss L t con in Eq. 5.12 L t total + = L t con .13 Update ψ t and W t by backpropagating the gradients of L t total .14 if (iter > ρ • I T S ) ∧ (t > 1) then 15Update prototype matrix µ t via Eq. ( 8).16 Calculate the noise estimation error loss L t dif f via Eq. (10) .Push W t into Q a and µ t into Q p : Q a ← W t , Q p ← µ t .Update θ t by backpropagating the gradients of L t dif f .Push the classifier weights with the best performance on the validation set of D t , denoted asẄt , into Q r : Q r ← Ẅt . 23 return Final ψ T , θ T , Q r .Huffpost (license: CC0: Public Domain) from [48] comprises 63, 907 news headlines from the Huffington Post, with the time span from 2012 to 2018. These news headlines belong to 11 categories: "Black Voices", "Business", "Comedy", "Crime", "Entertainment", "Impact", "Queer Voices", "Science", "Sports", "Tech" and "Travel". This dataset reflects changes in news content and style over time. Follwoing [45] , the first 4 years are used for training (T = 4) and the last 3 years are used for testing (K = 3). For each training domain, we randomly divide the data into training set and validation set in the ratio of 9 : 1.Arxiv (license: CC0: Public Domain) in [48] is a large-scale dataset, including 2, 057, 952 paper titles from 2007 to 2022. It reflects the change over time as research fields evolve. The task is to classify a research paper into one of 172 categories based solely on its title. For this dataset, we use data from the first 9 years as source domains (T = 9) and data from the last 7 years as target domains (K = 7). For each source domain, the data is randomly divided into training set and validation set in the ratio of 9 : 1.Input: sequentially arriving target domains T = {D T +1 , D T +2 , . . . , D T +K }, feature encoder E ψ T , conditional diffusion model E θ T , reference point queue Q r , number of categories C, batch size B, maximum diffusion step S, number of generated residual weights M g based on each reference point. ).1 for k = 1 to K do 2 Set D test = D T +k . 3 Calculate the prototype matrix µ test vai µ test [c] = 1 N test N test i=1 ptest i [c] • f test i , where ptest i = 1 |Qr| Ẅt ∈Qr softmax( Ẅt × f test i ), f test i = E ψ T (x test i ), c = 0, . . . , C -1.7Get the final label predictions on domain D test :{ŷ test i = argmax c p test i [c]} N test i=1, wherep test i = softmax( Wtest × f test i ). 8 return Label Predictions {{ŷ T +k i } N T +k i=1 } K k=1 .Yearbook (MIT license) dataset comes from [48] . It collects 37, 189 grayscale yearbook photos from 128 American high schools, with the time span from 1930 to 2013. The resolution of photos is 32 × 32. Photos from different years reflect changes in fashion trends and social norms over the decades. The task is to classify the genders from a yearbook photo. It is worth mentioning that we only use this dataset to evaluate the generalization performance of different methods in classification tasks. Following [45] , we group the data into domains at four-year intervals, resulting in 21 domains. And the first 16 domains are used as source domains (T = 16), with the last 5 domains as target domains (K = 5). For each source domain, we randomly select 90% of samples as the training split and 10% of samples as the validation split. And for each target domain, we evaluate on its all data.RMNIST (license: CC BY-SA 3.0) is constructed from MNIST dataset [7] which contains grayscale images of digits from 0 to 9. The image resolution is 28 × 28. RMNIST first randomly divides all data in MNIST into 9 groups and then creates 9 domains by rotating the 9 groups by 0 • , 10 • , . . . , 80 • , respectively. The rotation angle is used to simulate the evolving data distribution over time. Following [45] , we use the first 6 domains (T = 6) as source domains and the last 3 domains as target domains (K = 3). Similarly, we split each source domain into training and validation sets in the ratio of 9 : 1.fMoW (license: https://github.com/fMoW/dataset/blob/master/LICENSE) dataset is from [48] , which consists of 141, 696 RGB satellite images from 2002 to 2017. The visual features in these satellite images change over time due to human and environmental activities. The image resolution is 224 × 224 and the task is to classify the functional purpose of the buildings or land in a image into one of 62 categories. For this dataset, we consider each year as a separate domain. And the first 13 domains are used for training (T = 13), while the last 3 domains are used for testing (K = 3). The ratio of training data to validation data for each source domain is 9 : 1.2-Moons (license: https://github.com/BaiTheBest/DRAIN/blob/main/LICENSE) from [2] is constructed from 2-entangled moons dataset, where the lower moon with label 0 and the upper moon with label 1 contain 100 data points, respectively. 2-Moons creates 10 domains by counterclockwise rotating the 200 data points at an interval of 18 • . Similar to RMNIST, the rotation angle simulates the evolving of data distribution. Following [2] , the first 9 (T = 9) domains are used as source domain and the last domain is used as target domain (K = 1).Online News Popularity (ONP) (license: CC BY 4.0) in [2] summarizes a heterogeneous set of features related to the articles published by Mashable in a two-year period. This dataset is divided into 6 domains by time, and the goal is to predict whether an article is popular in social networks based on its features. Following [2] , we use the first 5 domains for training (T = 5) and the last domain for testing (K = 1). For the backbone of the task model, Huffpost and Arxiv apply pretrained DistilBERT base model [33] along with a bottleneck layer [20] to reduce the feature dimensions into 128. The bottleneck layer is implemented as the combination of a linaer layer, BatchNorm and ReLU. Yearbook uses the 4-layer convolutional network in [48] , RMNIST adopts the ConvNet in [29] , and fMoW employs the ImageNet-pretrained DenseNet-121 [15] along with a bottleneck layer [20] to reduce the feature dimensions into 256. Meanwhile, 2-Moons uses a MLP with two hidden layers of hidden size 64 and 128, and ONP adopts a MLP with one hidden layer of hidden size 128.For the conditional diffusion model, we implement it in a U-Net architecture similar to LDM [31] and make some modifications to better suit our method. Detailed modifications can be found in the code provided in the supplementary material. In Table 7 , we provide detailed configurations for U-Net E θ on different datasets. Please refer to original paper [31] for the meaning of different hyperparameters.Training details on different datasets are given in Table 8 , where B is the batch size for the task model, I DM is the inner iterations for updating E θ , λ is the loss tradeoff hyperparameter, ρ is the warm-up hyperparameter, L is the maximum length of the reference point queue Q r , M is the maximum length of the anchor point queue Q a and prototype queue Q p , and M g is the number of generated residual classifier weights based on each reference point. All experiments are conducted using the PyTorch packages and run on a single NVIDIA GeForce RTX 4090 GPU with 24GB memory.In Fig. 4 (a), we test the sensitivity of W-Diff to the loss tradeoff hyperparameter λ, the maximum length L of the reference point queue Q r and the number M g of generated residual classifier weights based on each reference point, where λ ∈ {0.1, 0.5, 1.0, 5.0, 10.0, 50.0}, L ∈ {1, 2, 4, 8}, M g ∈ {8, 16, 32, 64, 128}. We find that larger M g results in more weights for ensemble and seems to be better. W-Diff is a little bit sensitive to λ and L. Empirically, λ = 10.0 and larger L work well. In Fig. 4 (b), we plot the loss curve of L t dif f , when the conditional diffusion model is incrementally trained on the second source domain. Form the results, we see that the noise estimation error loss L t dif f steadily decreases and finally converges, demonstrating that using the FIFO queue to cache the recent M classifier weights and prototype matrices after the warm-up stage is feasible for the diffusion model training. Storing all checkpoints after the warm-up stage provides lost of training data for diffusion model but requires more storage cost. By contrast, using a FIFO queue with a fixed length balances the storage cost and the diversity of training data.In Table 9 , we list the model size and inference time of the conditional diffusion model on different datasets. Since the diffusion model only models the evolving pattern of the classifier weights which are the parameters of a linear layer, the model size that is measured by the number of parameters is small on all datasets. Besides, the inference time when forwarding the diffusion model for 1000 times to generate a batch of residual classifier weights is moderate. Concretely, forwarding the condition diffusion model for one time requires ≤ 181 ms. Certainly, acceleration techniques, e.g., DDIM [35] can be used to further reduce the inference time.E. 4 [45] , where a significance level of 0.05 is adopted.To comprehensively evaluate the effectiveness of W-Diff, we conduct the significance test (ttest) on Huffpost, Arxiv, Yearbook, RMNIST and fMoW datasets. Concretely, a significance level of 0.05 is applied. If the p-value is less than 0.05, then the accuracy difference between EvoS [45] and W-Diff is statistically significant. For clearer explanation, the -log(p) of each pvalue is plotted. In Fig. 5 , the majority of the -log(p) of the performance comparison between EvoS [45] and W-Diff are larger than -log(0.05), which means that W-Diff is statistically superior to EvoS [45] at most datasets.• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: **Summary**

This paper proposes a novel method called Weight Diffusion (W-DiFF) for EDG. The approach leverages a conditional diffusion model to learn the evolving patterns of classifiers during domain-incremental training, allowing the model to generalize more effectively to unseen future domains. The method is validated through extensive experiments across various datasets, including text, image, and multi-variate classification tasks.


**Decision**

The decision to accept is based on the proposed method’s ability to tackle a challenging and practical setting of EDG, where source domains are not simultaneously available. This represents an advancement over existing approaches that typically assume all source domains are available concurrently.  Additionally, the method introduces an innovative application of a conditional diffusion model to learn evolving patterns in classifier weights, which, while subject to some debate among the reviewers, adds a novel perspective to the field of domain generalization. The strong performance across multiple benchmarks, combined with the comprehensive rebuttal that addressed concerns (e.g., computational cost) and provided further validation, supports the acceptance of this paper.

For the final version, please ensure that the additional experiments and comparisons provided during the rebuttal are included, as they significantly strengthen the paper's contribution.

Reviews: review: paper_summary: paper_summary: This paper deals with the problem of evolving domain generalization in non-stationary environments, where dynamically changing source domains arrive sequentially, but we only have access to data samples from the current domain (and not the past ones). The main idea is to learn a conditional diffusion model that predicts how the evolving pattern of the classifiers, as the source domains keep changing. In a nutshell, the conditional diffusion model learns how to go from past classifier weights (reference points) to the weights of the current classifier (anchor point), conditioned on prototypes of the current domain. One challenge concerns the shared feature encoder: if the encoder is updated using the latest source domain, then it will overfit to it. To avoid this, the authors suggest to enforce prediction consistency among multiple classifiers, so that all classifiers from past and present domains give the same prediction given the same feature representation. This implies that the past reference points do not become obsolete. During inference, the goal is to predict the classifier weights given the current domain prototypes. For this purpose, the framework uses the conditional diffusion model to cheaply generate a large number classifiers, which it then averages for improved robustness. Extensive experiments on synthetic and real datasets show strong performance for the proposed framework.
strength_weakness: strength_weakness: - In inference phase, the authors assume they know the dataset (without the labels) for the future timesteps $\{T+1,\dots,T+K\}$. Of course, this cannot work when the datasets are not given. In such a case, the model must also be in a position to infer the future data distribution for the points $x_i$, in order to then estimate the prototypes $f_{test}$. The current framework seems not to be able to deal with that. I think it would help if the authors explain what the limitations of their framework are, and why for instance it is very difficult to predict the future data distribution in non-stationary environments. The current weight-diffusion model assumes that we can have access to future data but not their labels during inference, but this may not always be the case.
- I like the fact that the authors condition on the context. But I was not clear why the context only consists of the historical weights and the current prototypes. Why not also include the timestep difference between the reference point and the current timestep? For instance, assume we are given the context (reference point + prototype). If the reference point is from 5 timesteps ago, then the predicted weights might be different compared to the case where the reference point is from 10 timesteps ago, even if the reference point value is the same. If we are not given how far into the past the reference point lies, then I was thinking that predicting the current weights is harder. On the other hand, perhaps this is already taken into account implicitly, in the framework, but this is not immediately clear.
- The point above also points to the limitation of the current framework, which is a lack of formal theory. It works very well  experimentally, but some aspects are not very clear.
- There are numerous typos and bad grammar throughout the paper. The authors should do a very careful proofreading and fix all the errors.
questions: questions: - Why is it not necessary to condition on the timestep difference between the reference point and the current anchor? Intuitively, one might expect that knowing how far into the past the reference point is could lead to better predictions. Did the authors try this?
- I was under the impression that U-Net still needs the parameters $\beta_s$ in Equations (3) and (4).. If yes, how did the authors set these hyperparameters? Where they fixed throughout training?
- I am not sure I can understand Figure 3a. $\tilde{W}^{7\mid 5}$ is concentrated in the upper left, whereas $W^7$ is at the very bottom. And, yet, both of them perform very well in Figure (3b). What does that exactly say about the computed weights? Maybe it means that the visualization does not really tell us anything about test performance?
- In the ablation study in Section 5.3, what is Variant B?
- Can the authors be more clear about the limitations of their framework? Could their framework be modified to also predict the future data distribution (e.g., how the $x_i$ will be distributed in the next timestep)?
limitations: limitations: No concern.
paper_summary: paper_summary: This paper presents Weight Diffusion (W-Diff), a framework for domain generalization in non-stationary environments. W-Diff leverages a conditional diffusion model in the parameter space to learn the evolving pattern of classifiers during domain-incremental training. Experiments on synthetic and real-world datasets demonstrate its superior generalization performance on unseen future domains
strength_weakness: strength_weakness: **W1.**
The paper does not sufficiently justify why a diffusion model is chosen for the domain generalization (DG) problem. While diffusion models have shown excellent performance in generation tasks, the specific advantages they offer for DG are not clearly articulated.

**W2.**
The computational complexity of the diffusion model training might be a barrier for very large datasets or real-time applications. A thorough analyses on the computational complexity espeically for training is needed.

**W3.**
The evaluation is limited to classification tasks; applicability to other types of tasks, such as regression, remains unexplored.
questions: questions: Please refer to my weaknesses section.
limitations: limitations: The authors discussed some limitations of this work in their paper.
paper_summary: paper_summary: The paper proposes a novel method called Weight Diffusion (W-Diff), which employs a conditional diffusion model in the parameter space to learn the evolving patterns of classifiers during domain-incremental training. During inference, the proposed method uses an ensemble of classifiers tailored to the target domain for robust predictions.
strength_weakness: strength_weakness: 1. The experiments are conducted on relatively small networks, and the diffusion model is used solely to generate the classifier. It remains unclear whether this method can scale to larger networks.
2. The paper should include a comparison with Variational Autoencoders (VAEs) or other generative models. Given that diffusion models are more complex and challenging to train, it is important to demonstrate the necessity of using them.
3. The paper should include a comparison of training time with state-of-the-art algorithms.
4. It would be good to include a notation table in the appendix, as there are numerous notations used throughout the paper.
questions: questions: See Weaknesses.
limitations: limitations: The limitation of the paper is that it only considers generating the classifier. It would be interesting to explore whether it can generate the entire network.
paper_summary: paper_summary: This manuscript aims to tackle the evolving domain generalization (EDG) issue, namely the domain gradually evolves in an underlying continuous structure. The paper introduces the idea of Weight Diffusion (W-Diff), a conditional diffusion model in the parameter space to learn the evolving pattern of classifiers. Combining such types of classifier with weight ensembling and a domain-shared feature space allows robust prediction. The effectiveness of the proposed method is examined on two text classification datasets, three image classification datasets, and two multi-variate classification datasets.
strength_weakness: strength_weakness: * Unclear novelty. The idea of using a diffusion model to generate model weights/heads appeared in [1, 2]. However, the manuscript did not cite and discuss these two papers (and maybe their follow-up works), making the exact contribution unclear. From the reviewer's perspective, idea 1 of using the conditional diffusion model to model parameter evolution patterns is interesting and intuitive; idea 2 of learning domain-shared feature encoder is standard. Combining idea 1 and idea 2 and applying it to the area of evolving domain generalization is ok, but still needs a careful discussion and ablation study.
* The performance gain over different datasets looks marginal.
* The manuscript needs more ablation studies. E.g., What if instead of generating the classifiers on the fly for the unseen domain, we just leverage the past classifiers for the ensemble prediction, or test-time adaptive classifier ensembling as in [3]?


### Reference
[1] Learning to Learn with Generative Models of Neural Network Checkpoints. https://arxiv.org/abs/2209.12892

[2] Diffusion-based Neural Network Weights Generation. https://arxiv.org/abs/2402.18153

[3] Adaptive Test-Time Personalization for Federated Learning. https://arxiv.org/abs/2310.18816
questions: questions: NA
limitations: limitations: NA

Comments: title: Thanks for positive feedback
Further response to concerns about the computational cost
thank you for response
Thanks for positive feedback
Discussion period starts
Thanks for upgrading score
Thank you for the rebuttal
comment: We sincerely appreciate your valuable reviews and positive feedback. We hope the idea and approach presented in this work can inspire more studies in this direction.
Thanks for your feedback and we are happy to know that our rebuttal has addressed most of your concerns. As for the computational cost during the training process, we would like to provide further clarifications.

**Firstly**, the problem we focus on is the evolving domain generalization (EDG) in the domain-incremental setting, which is an under-explored area, compared with previous EDG. The consideration of domain-incremental setting mimics the dynamics of training domains in the real-world, which is more practical yet challenging. The resulting benefit is that once a new training domain is available, we can incrementally train the model only on the new domain, instead of training from scratch with all old domains and the new domain using previous non-incremental EDG methods. The latter would be inefficient when new domains continually emerge with the passage of time.

**Secondly**, the training time we report in previous response to **Q2** is the total training time when the number of source domains is $T$. In the following table, we further compared the average training time per domain and the performance improvement over the baseline GI. Though our W-Diff indeed requires more training time per domain, we think the increment of training time is acceptable, considering the significant performance improvement over GI. Besides, if a new source domain of similar dataset size comes, W-Diff just requires roughly the average training time to train the model on the new source domain.

-------------------------------Computational cost and performance comparison on **RMNIST** dataset (T=6, K=3)-------------------------------
| Method | Total training time (h) | Average training time per domain (h) | GPU memory (GB) | $\mathcal{D}^{T+1}$ accuracy (%) | OOD avg. accuracy (%) | OOD worst accuracy (%) | 
| :-: | :-: | :-: | :-: | :-: | :-: | :-: |
| GI | 2.40 | 0.40 | 1.9 | 97.78 | 91.00 | 82.46 |
| W-Diff | 4.61 | 0.77 | 4.0 | 98.70 | 94.12 | 87.36 |
| Increment $\Delta$ of W-Diff over GI| 2.21 | 0.37 | 2.1 | **0.92** | **3.12** | **4.90** |

-------------------------------Computational cost and performance comparison on **Huffpost** dataset (T=4, K=3)-------------------------------
| Method | Total training time (h) | Average training time per domain (h)| GPU memory (GB) | $\mathcal{D}^{T+1}$ accuracy (%) | OOD avg. accuracy (%) | OOD worst accuracy (%) | 
| :-: | :-: | :-: | :-: | :-: | :-: | :-: |
| GI$^\clubsuit$ | 10.04 | 2.51 | 17.3 | 64.96 | 63.11 | 60.15 | 
| W-Diff | 12.61 | 3.15 |15.6 | 73.91 | 72.29 | 70.40 |
| Increment $\Delta$ of W-Diff over GI | 2.57 | 0.64 | **-1.7** | **8.95** | **9.18** | **10.25** |

*(GI$^\clubsuit$ denotes that a much smaller batch size is used due to the **GPU memory explosion** when using the method on Huffpost dataset with the backbone of DistilBERT base.)*

**Thirdly**, it is noteworthy that GI requires a pre-training stage on all domains and then sequentially finetunes the pre-trained model on each domain. That is, if a new source domain comes (i.e., $T$ changes), it needs to re-execute the pre-training stage from scratch. By contrast, our W-Diff incrementally trains the previously trained model on the new source domain, rather than training from the first source domain every time. Indeed, in the fixed $T$, our W-Diff is at a disadvantage in terms of training time. But in practice, **in the long run, our method is more advantageous in training time**. To verify this, we record the total training time of GI and W-Diff on RMNIST dataset, when $T$ increases in a sequence of $2, 3, 4, 5, 6$. Concretely, the result is **`6.74h (GI) vs 4.61h (W-Diff)`**. Here, the training time of W-Diff is the same as in the above table, because the training procedure of W-Diff has already simulated the increasing of $T$.

Certainly, further improving the training efficiency of W-Diff is a nice direction for future work. We hope that our response could mitigate your concerns about computational cost. Looking forward to your feedback. Thanks so much!
I thank the authors for their rebuttal. I will keep my original score for now (which anyway leans towards acceptance), but may revise my score upward in the next phase.
Thank you so much for your positive feedback and dedicated time to review our paper! We are glad to know that our rebuttal and new experiments have addressed your concerns.
I appreciate the author's responses, which addressed most of my previous concerns, though the computational cost associated with the use of the diffusion model remains a consideration. I will maintain my current score of 5 and take the rebuttal into account in the next phase of discussion.
Dear reviewers,

Thank you for your valuable contributions to the NeurIPS review process! The author-reviewer discussion period has now begun. I’ve noticed that the ratings for this paper are a little dispersed, and opinions among the reviewers are not fully aligned. This makes our discussion even more crucial to reach a consensus and ensure a fair and thorough evaluation. Please engage actively with the authors during this period. If you have any questions or need further clarification on any points, this is the best time to address them directly with the authors.

best,

AC
Thank you very much for your detailed reply. My concerns are addressed and I will keep my score towards acceptance.
We really appreciate your feedback and are pleased that our rebuttal has addressed your concerns, leading to a raised rating! In future revisions, we will update the manuscript according to your suggestions.
The reviewer thanks for the detailed responses and additional results. The reviewer decided to raise the score to 5.