Paper: Incentivizing Data Collection from Heterogeneous Clients in Federated Learning (ICLR-2024)

Reviewer comment (Experiments):
There seems to lack a comparison with existing methods or simple baselines.

Author response:
**Response to W4**: Thank you for pointing out this. To our knowledge, there currently aren't any existing methods or baselines available that we can use for comparison in our study. We would be grateful for any recommendations you could provide for suitable baselines.

Reviewer comment (Writing):
The abstract mentions "decentralized agents", but the setting does not seem decentralized since there is a central learner.

Author response:
**Response to Q1**: Sorry for any confusion. "Decentralized clients" in the context of federated learning or distributed systems refer to individual nodes, devices, or entities that operate independently of a central authority or location. Each client holds its own dataset and performs computations on this data locally. To prevent any misunderstandings in the revised version, we will refrain from using the term “decentralized”.

Reviewer comment (Writing):
In introduction,  
> Hence the correct way to provide incentives for agents is to encourage agents to upload models that can capture the data heterogeneity.  
Is this made precise somehow?

Author response:
**Response to Q2**: Thank you for pointing out this. We will revise it as “Hence, the appropriate method to incentivize agents should take into account not only the size of the sample, but also the potential data heterogeneity.”

Reviewer comment (Novelty):
In introduction,  
> Such a solution is necessary for designing incentive mechanisms in a practical FL system but currently missing from all existing work ...  
Precisely, how is the necessity of such incentive mechanisms demonstrated? For instance, is it shown that all existing works perform sub-optimally in some way?

Author response:
**Response to Q3**: A common limitation in existing studies is their focus on measuring each agent's contribution by the number of samples used, thereby encouraging the use of more data samples. These approaches tend to overlook the impact of data heterogeneity. Solely using sample size to evaluate contribution may not effectively reflect the quality of the data, especially in federated learning where data heterogeneity largely affects model convergence. Hence, an incentive mechanism that accounts for data heterogeneity is essential. 
From the learner's perspective, our proposed mechanism facilitates more optimal model convergence compared to existing methods. For example, at the same sample size level, our model's convergence is significantly faster compared to existing mechanisms. This improved performance is largely due to our approach effectively addressing data heterogeneity. This is evidenced by the performance results shown in Figure 1.

Reviewer comment (Theory):
In introduction,  
> We are the first to prove that Wasserstein distance is an essential factor in the convergence bound of FL with heterogeneous data (Lemma 4.2)  
If my understanding is correct, the Wasserstein distance is used to derive a specific instantiation of an existing result, which already provides a way to formalize the heterogeneity of data in FL. It is unclear why Wasserstein distance is essential.

Author response:
**Response to Q4**: We admit that the notion of the Wasserstein distance has been applied in various fields, not just limited to federated learning. However, to our knowledge, there is no existing similar work that has specifically applied the Wasserstein distance to specifically assess the impact of data heterogeneity on the convergence bound in federated learning.
In federated learning, "label shift" is the most common non-iid setting, which refers to a scenario where the distribution of labels (or output classes) in the training data varies across different clients over time. Therefore, the Wasserstein distance serves as a practical and important metric for measuring the label distribution, particularly in classification problems. This distance measures how much one distribution needs to be altered to resemble another, making it highly relevant for evaluating discrepancies in label distributions across different datasets. Apologies for any misunderstanding regarding the use of 'essential'. We will replace it with 'important'.

Reviewer comment (Theory):
Is the Wasserstein distance formally recalled somewhere? Is the definition of $\delta_k$ the formal definition? If so, does it mean that it does not work for regression problems?

Author response:
**Response to Q5**: Limited by paper space, we slightly introduce the Wasserstein distance in the definition of $\delta_k$.The Wasserstein distance is a well-known measure of the distance between two probability distributions. It is a concept derived from the field of optimal transport, where it represents the minimum "cost" required to transform one distribution into another. This metric provides a more intuitive way to compare distributions, consistent with our objective to reduce the degrees of non-iidness by adding additional samples.The 11-norm form of the Wasserstein distance is formally defined as $W_1(P, Q)= \inf _{\gamma \in \Gamma(P, Q)} \int\_{X \times Y} \|x-y\|_1 d \gamma(x, y)$, where $\|x-y\|_1$ is the l1-norm, calculating the absolute difference between points $x$ and $y$ in the distributions. We simply adapt this into a discrete distribution format, as illustrated in the definition of $\delta_k$.
In our works, we mainly focus on classification problems, commonly encountered in federated learning. Therefore, the formulation of our $\delta_k()$ is defined over "I", which represents the number of classes in a classification problem.

Reviewer comment (Reproducibility):
After the definition of $\delta_k$,  
> p(c) indicates the reference (iid) data distribution in the centralized setting  
Why is there a centralized setting, and what is meant by iid data distribution? Furthermore, how to obtain $\delta_k$ in implementation?

Author response:
**Response to Q7**: This centralized setting refers to a scenario where data is aggregated in a single, central repository, in contrast to the distributed nature of federated learning where data is scattered across multiple clients. Typically, we would use the distribution of this ideal centralized setting $p^c$ as the iid distribution for federated learning and a benchmark to measure the degrees of non-iidness in clients' data, providing a reference point for comparing the data distribution across various decentralized clients. 
Owing to privacy concerns, accessing the iid distribution across agents is not possible. Thus, in federated learning, we often use the distribution $p^c$, derived from an ideal centralized setting, as the representative iid distribution. This serves as a standard for measuring the degrees of non-iidness in clients’ data, offering a basis for comparison across these distributed data sources. In practice, $p^c$ typically reflects the test distribution for performance evaluation from the learner's standpoint.

To assess the degree of non-IIDness in an agent's local training data without it being explicitly provided, one potential approach is to use statistical techniques to analyze the data's distribution. Given the definition of $\delta_k = \frac{1}{2}\sum_{i=1}^{I}|p^{(k)}(y=i) - p^{(c)}(y=i)|$, in practice, we can reasonably assume that statistical information of local datasets (especially label proportions $p^{(k)}(y=i), \forall i \in [I]$), is accessible to the learner without compromising privacy. Also, the learner can disclose the label proportions of the reference dataset or distribution  $p^{(c)}(y=i), \forall i \in [I]$ to agents. This act effectively communicates a preference for iid settings to the agents. These assumptions allow for a straightforward assessment of the degrees of non-IIDness in the data.

Reviewer comment (Writing):
In Section 3.2,  
> Learner: seeking to train a classifier that endeavors to encourage agents to decrease the degree of non-iid. Thereby, attaining faster convergence at a reduced cost.  
What is meant by "decreasing the degree of non-iid"? Furthermore, precisely what is the cost in the "at a reduced cost"?

Author response:
**Response to Q8**: "Decreasing the degree of non-iid" implies that agents are incentivized to modify their local datasets to resemble more iid-like distributions. Specifically, this can involve incentivizing agents to contribute more data from minority classes instead of majority ones in their datasets. "A reduced cost" refers to the learner's aim to minimize the expenses incurred while motivating agents to decrease the non-iid degree of their data, thereby maximizing the learner's payoff.

Reviewer comment (Writing):
What are $ g_t, \bar{g}_t, f^* $ in Lemma 4.1?

Author response:
**Response to Q9**: Apologies for missing explanations on these parameters. We will fix this in the revision.
$f^* = f(w^*)$ indicates the optimal empirical risk.
Then, $ \overline{\mathbf{g}}\_t = \sum\_{k=1}^N p\_k \nabla F\_k(\mathbf{w}_t^k) $ represents the weighted sum of gradients of loss functions $F_k$ for each client $k$. Here, $\mathbf{w}_t^k$ are the model parameters for client $k$ at time $t$, and $p_k$ is the weight for each client.
Similar to $\overline{\mathbf{g}}\_t $, $\( \mathbf{g}\_t = \sum\_{k=1}^N p\_k \nabla F\_k(\mathbf{w}\_t^k, \xi\_t^k)$ includes a stochastic component $\xi\_t^k $, which typically represents a stochastic component such as a minibatch of data. In practice, $ \mathbb{E}\mathbf{g}_t = \overline{\mathbf{g}}_t $.

Reviewer comment (Theory):
The learner is assumed to have access to $\mathcal{D}_c$, what are its properties and how can this assumption be satisfied?

Author response:
**Response to Q11**: Actually, $D_c$ is an auxiliary validation set, following the test set's data distribution. It's common to assume access to such a validation set for evaluating model performance. In our study, this set is solely used for evaluation purposes, meaning no extra information is introduced by incorporating the validation set.

Reviewer comment (Writing):
What exactly is $\text{Bound}(e)$?

Author response:
**Response to Q12**: $\mathrm{Bound}(e) $ denotes the model's performance upper bound under agents' efforts $e$. For example, $\mathrm{Bound}(e) $ represents the upper bound as illustrated in Example 1.  More details can be found in the definition of payoff functions.

Reviewer comment (Writing):
What is the "Ninety-ninety rule"?

Author response:
**Response to Q13**: The ninety-ninety rule is a humorous aphorism that states: "The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time."

Reviewer comment (Theory):
In Definition 5.1, how are the constants $d_k^1, d_k^2$ set or determined, and how are $d_k^1, d_k^2$ related for a fixed $k$?

Author response:
**Response to Q15**: Based on Assumption 5.1, we can determine the maximum and minimum values of $\partial f\left(e_k, e_{k^{\prime}}\right) / \partial e_k$ when $e_k=0$ and $e_k=1$, respectively. These extremal values correspond to the constants $d_k^1$ and $d_k^2$. More details can be found in the proof of Lemma B.1 of Appendix B.3.

Reviewer comment (Theory):
Why (only) Nash equilibrium in a Stackelberg game? Can something be said about the behavior of the learner (i.e., leader in the Stackelberg game) to consider the Stackelberg equilibrium?

Author response:
**Response to Q16**: In Remark 1 of Appendix B.3, we additionally explore the existence of other common equilibriums in our context, including no one participating and free-riding. Briefly speaking, the Stackelberg game in game theory is a model describing a strategic interaction where players move sequentially rather than simultaneously. It consists of a leader and one or more followers, where the leader must anticipate how followers will respond to maximize its payoff, while followers act to maximize their own payoffs given the leader's action. Hence, the structure of the Stackelberg game is indeed well-suited to address the incentive issues in federated learning.

Reviewer comment (Experiments):
In experiments under "Parameter analysis",  
> Recall that we set the learning rate $\eta = 0.01$.  
Is it for all experiments?

Author response:
**Response to Q17**: Yes.

Reviewer comment (Presentation):
How should Figure 3 be interpreted? In particular,  
> the utilities of two randomly selected agents remain stable, which indicates an equilibrium.  
Which are the two agents? and the fluctuations (which the authors attribute to the random selection of the agents) make it difficult to see the equilibrium.

Author response:
**Response to Q18**: Sorry for this typo. From the theoretical aspect [Theorem 5.3], a Nash equilibrium in our settings means that no agent can improve their utility by unilaterally changing their invested effort levels. Hence, the utility of agents could be viewed as an indicator of the equilibrium $\rightarrow$ if the utility of agents remains almost constant as the training progresses, an equilibrium is reached. As shown in Figure 3, the utilities of two randomly selected agents remain stable, indicating equilibrium. 
We can attribute the observed fluctuations to randomness for two reasons. Firstly, Figure 4 demonstrates that agents can achieve maximum utility despite randomness, indicating the utility function's stability. Secondly, in Remark 2 of Appendix B.3, we also discuss the consistency between randomly selecting one peer agent and the extension that takes an average over all other agents. Ideally, this would result in the utility curves being flat, represented as horizontal lines.

Reviewer comment (Theory):
While there are a number of key assumptions made throughout the paper, an important one is "equilibrium shall prevail when the impact of unilateral deviations in an agent's effort level remain limited to their own utility." and an equilibrium solution may not exist if small changes affect other agent's utilities. Could the authors discuss whether adversarial behavior falls under this category of assumptions and how can the system potentially protect itself from such cases.

Author response:
Indeed, the above-mentioned statements are mild assumptions that can be reformulated as properties of utility functions. It's quite common to assume that utility functions are well-behaved, such as being convex or smooth, when analyzing Nash equilibrium [1, 2, 3]. For instance, in Definition 5.1, we formulate them as the bounds on the first-order gradient of the utility function, which are conditions that are generally easy to meet. Given this framework, discussing adversarial behaviors becomes less pertinent.
Indeed, the Nash equilibrium acts as a safeguard against adversarial behaviors. It ensures that agents can maximize their utilities exclusively by exerting the optimal level of effort. This aspect highlights a significant advantage of the Nash equilibrium concept.

Reviewer comment (Experiments):
A recent trend in federated learning involves the use of pre-trained weights, either from ImageNet pretraining or foundation models, to close the gap in performance between IID and non-IID scenarios. Given that such choices directly affect the prediction ability of agents, with each agent having a unique understanding of the dataset based on the training setup and choice of pre-trained model, could the authors comment on pre-trained weights suppressing the underlying heterogeneity of local data distributions and how this affects the proposed framework?

Author response:
Thanks for this great question! Utilizing pre-trained weights in federated learning effectively suppresses the impact of data heterogeneity, resembling warm-start strategies or personalization. However, the influence of this heterogeneity could be reserved during continued training from these weights. It's important to highlight that theoretical analysis serves as the foundational starting point for our work. The theoretical framework might differ in the context of pre-trained weights. To integrate this scenario into our proposed model, we might need to introduce certain assumptions regarding the pre-trained weights.

Reviewer comment (Theory):
The model is not clear/justified what does the "effor" mean for agents? How can agents narrow down the non-iid degree of their own data simply by incurring more "effort"? I don't think such model of effort and non-iid level is reasonable and realistic.

Author response:
In our paper, the term "effort $e_k$" refers to the level of effort exerted by a client in collecting additional data samples, with the primary objective of maximizing utility. 
In fact, agents can decrease the non-iid degree of their data by contributing their data with a preference for an IID setting. For example, agents can be incentivized to provide data samples for minority classes rather than for majority classes in their local datasets. 
It's common and rational to incentivize agents to offer more/high-quality data, known as data sharing. From the perspective of data-sharing, agents may exert effort to supply more data samples [1, 2]. Our work advances the field by establishing a preferred method for data collection in data-sharing.  Therefore, from other perspectives, agents are incentivized to use their data strategically rather than merely contributing all their local data immediately.

Reviewer comment (Theory):
The authors assume complete information, e.g., agents know the true central distribution, this is also not reasonable/justified.

Author response:
Though it seems less practical than the incomplete game, studying complete information games is a standard and key practice that provides vital theoretical insights into strategic interactions. Besides, the complete game is reasonable because most of the complete information is available [1, 2]. In fact, agents only know the class proportion of the true central distribution, rather than full information about the true central distribution. Therefore, we believe that accessing the statistics information of the true central distribution is a mild and reasonable assumption. For example, participants generally know the upper bounds of convergence and generalization loss, with only the marginal cost remaining private. This setup also makes sense when costs, like those for data creation (e.g., autonomous driving) or labeling, can be readily estimated by all parties.

Reviewer comment (Novelty):
Definition 5.1 is quite similar to Definition 6 in Blum et.al, 2021, but it is not cited when Definition 5.1 is proposed.

Author response:
Thank you for pointing out this. We will fix this in the revision.

Reviewer comment (Presentation):
Not clear why Figure 3 indicates equilibrium.

Author response:
From the theoretical aspect [Theorem 5.3], a Nash equilibrium in our settings means that no agent can improve their utility by unilaterally changing their invested effort levels. Hence, the utility of agents could be viewed as an indicator of the equilibrium $\rightarrow$ if the utility of agents remains almost constant as the training progresses, an equilibrium is reached. As shown in Figure 3, the utilities of two randomly selected agents remain stable, which indicates an equilibrium. Note that the existing fluctuation of utility mainly results from the randomness of selecting peers.