Title: Incentivizing Time-Aware Fairness in Data Sharing
Venue: NeurIPS 2025
Accepted: yes
Keywords: Cooperative Game Theory

Incentives

Data Sharing
Abstract: In collaborative data sharing and machine learning, multiple parties aggregate their data resources to train a machine learning model with better model performance. 
However, as the parties incur data collection costs, they are only willing to do so when guaranteed incentives, such as fairness and individual rationality. 
Existing frameworks assume that all parties join the collaboration simultaneously, which does not hold in many real-world scenarios. 
Due to the long processing time for data cleaning, difficulty in overcoming legal barriers, or unawareness, the parties may join the collaboration at different times. 
In this work, we propose the following perspective: 
As a party who joins earlier incurs higher risk and encourages the contribution from other wait-and-see parties, that party should receive a reward of higher value for sharing data earlier. 
To this end, we propose a fair and time-aware data sharing framework, including novel time-aware incentives.
We develop new methods for deciding reward values to satisfy these incentives. 
We further illustrate how to generate model rewards that realize the reward values and empirically demonstrate the properties of our methods on synthetic and real-world datasets.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work considers the problem of heterogeneous arrivals of data contributors. It proposes to provide higher incentives to the ones contributing earlier to motivate participation. The authors proposed two methods to determine the reward values.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The heterogeneous time arrival problem is interesting.\n- The analysis of this work is complete. \n\nWeakness: \n- The data was not actually characterized in this work. Specifically, whether non-IID or IID data is considered? If non-IID, then how do the data distributions of different parties affect the reward? If IID, then only the data quantity matters, i.e., no need to distinguish between parties. Further, if the data distribution does not matter, then it basically means the modeling is not related to the data themselves. In this case, the proposed methods should be able to apply to other scenarios without modifications. This is not good as the main feature of data sharing was not characterized. The modeling is too general.\n- F1 and F2 are basically the same concept. People usually use individual rationality without mentioning non-negativity.\n- In F5, \"improving model quality\" is non-rigorous. As mentioned earlier, the data together with the model was not characterized in the formulation.\n- F7 and F8 are human-invented properties. A more formal way should be formulating a problem and then showing that earlier participation should impose a higher reward. However, in this work, the authors impose the setting that earlier participation should have a higher reward. This may not be optimal sometimes. \n- The approach proposed in this work is relatively standard. Again, it is not actually related to data sharing, because those key features were not characterized in the formulation. The proposed approaches are simply existing approaches in coalition games without much modification associated with data sharing. \n- How is gamma tuned?\n- The approach should be compared with existing works. Although there may not be works considered similar settings, some incentive works should be compared with to show the enhancement for addressing heterogeneous arrivals. \n- Fairness is not modeled in this work.\n\nQUESTIONS:\n-  How is the data modeled? How does the data distribution affect model quality? This is important because this work emphasizes data sharing rather than game design. \n- Why is it reasonable to set higher rewards for earlier participation? I understand the intuition, while a formal mathematical formulation is needed to motivate this setting. That is, there should be an objective with physical meaning, solving which we can obtain that \"higher rewards for earlier participation\".\n- How is fairness modeled in this work? Fairness should be a formal mathematical metric rather than an intuition.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-23"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles the problem of incentivizing parties in collaborative machine learning who join at different times. Unlike prior work assuming simultaneous participation, the authors introduce a time-aware fairness framework that rewards earlier contributors more, reflecting their higher risk and contribution to collaboration momentum. They propose two reward mechanisms—time-aware reward cumulation and time-aware data valuation—both designed to satisfy fairness and new time-sensitive incentive conditions. It is novel to effectively balance data quality and joining time, encouraging early, valuable participation.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Introduces a novel time-aware fairness framework for CML, addressing asynchronous participation—an underexplored but realistic setting. \n- Proposes two theoretically sound methods with formal guarantees for satisfying fairness and time-aware incentives \n- Well-structured with clear motivation, incentive definitions, and illustrative examples\n\nWeaknesses\n- The paper has already discussed some limitations, such as computational efficiency and the choice of a non-Federated Learning (non-FL) setting. Regarding computational efficiency, I agree that this issue is not unique to this work and therefore should not be considered a limitation specific to this paper. However, I have some questions about the non-FL setting, which I will raise later.\n- While I acknowledge the strong theoretical contributions of this paper, I would like to see more experiments on large-scale real-world datasets if time permits. I will elaborate on this point later.\n\nQUESTIONS:\n1. I am curious about how to set the parameters in practice to control the emphasis on joining times and early participation. A user may want to know these parameters to decide whether to submit noisy data early or spend more time cleaning and submit high-quality data later. \n\n2. I understand that it may not be possible to guarantee the strong theoretical properties of the reward values under the online FL setting. However, it would be helpful to see a simple extension of your method to this setting along with empirical results. This is important to confirm the practical applicability of your work. If time constraints prevent experimentation, a discussion of your plans to extend the approach to online FL would be valuable.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studied the problem of data sharing in collaborative machine learning. Traditional frameworks typically assume all the parties are joining the data sharing mechanism at the same time. However, different parties may have different preparation time, and earlier joiners suffer from higher risk. Motivated by these practical concerns, this paper proposed a new framework of data sharing, taking into the time-aware incentives into consideration.\n\nThe paper proposed new methods for deciding reward values to satisfy these time-aware incentives. The, the paper demonstrated how to generate model rewards that realize the reward values and empirically illustrate the properties of the proposed methods on synthetic and real-world datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The time-aware incentives are motivated by practical concerns, and are interesting and novel considerations. It is reasonable that the data valuation and reward realization should change if parties join the data sharing process at different times due to various reasons. Thus the paper  provides explorations on this less-studied aspect.\n - The paper proposed a detailed framework with eight incentive conditions. These conditions are illustrated in detail to incorporate the time-aware considerations. \n- Theoretically, the two proposed reward scheme satisfied the proposed list of incentive conditions. Empirical results further demonstrated that the two schemes comply with the incentive conditions.\n\nWeakness:\n- there is no comparison between the proposed time-aware reward scheme and classical reward schemes that do not take the time-awareness into account. Although it's shown that the proposed two schemes comply with time-aware incentives, the benefit comparing to existing non time-aware reward schemes is unclear\n\nQUESTIONS:\n- Is there any redundancy among the proposed eight time-aware reward incentives? Does complying with a subset of them imply certain compliance on others?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper looks at time aware incentives in collaborative learning. For instance, suppose a group of agents, each having their own data-set want to jointly train a model by pooling their datasets then we call this collaborative learning. Time-awareness looks at the aspect where the agents choose to participate (or not) at different time stamps. In such a setting, the authors look at rewarding agents based on two parameters: (1) The quality of the data contributed by the agent and (2) The time at which the agents choose to participate -- Early contributions should be incentivized (while ensuring that late, high-quality data contributions are also incentivized).\n\nTo this effect, the authors set up eight desirable properties that an incentive should satisfy. Out of these, five have been established in previous works on non-time-aware fairness in collaborative learning. They introduce three new that essentially state that agents who have same quality datasets, with one agent joining before the other agent, the one that joins earlier should get higher reward -- provided the agent's data is not useless. They also have a property that says that if an agent's data is highly valuable (which they call necessary) this agent should get a high reward irrespective of when she joins the collaboration.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1) The authors have defined and addressed an important problem that has surprisingly not been studied before.\n2) The set of properties the authors define are very reasonable (although I have some questions about this stated below).\n3) The paper is well-written and easy to follow. I checked proofs for one section and they seemed correct to me.\n\nWeaknesses:\nFrom the technical side, the paper seems pretty direct. At the end of the section where the properties are defined, the authors demonstrate how Shapley value fails to provide rewards that satisfy the properties outlined (which is itself not surprising) however, their incentives are eventually based on Shapley value. For instance, one reward model is computing the Shapley value at each time point based on only the subset of agents that have joined until then and then takes a convex combination of these values. This seems a relatively straightforward extension.\n\nHowever, given that the model is new, highly relevant to the real-world setting, I would say that overall I think this is a decent submission to NeurIPS.\n\nQUESTIONS:\n1) The way Property F6 is defined, it seems that if agents i, j have the same data set that is necessary to the other agents (but not necessary to each other i.e., it suffices for either i or j to join the collaboration to get a non-zero value model) and i and j join at different times, they get an equal reward -- shouldn't a more reasonable requirement be that the i gets a higher reward? \n\n2) On line 189, you have a sentence in bold that says that \"a party get higher reward by waiting longer to curate a better data set due to F(6)...\" however, F(6) is very strong and this will be true only if the data set contributed by the agent who waits longer is the only data set that provides any non-zero value. This seems rather stronger than what the statement is claiming -- simply better datasets that join later may not be compensated highly -- this will depend (in a complicated fashion) on how much \"better\" the data set is and how the parameters \\beta (or \\gamma) are tuned. Is this correct?\n\n3) Can you elaborate more on the aspect of rewards -- as far as I understand, you are rewarding agents with models whose accuracy is proportional the the rewards you get. If this is the case then the fact that you can compute a model with any desired accuracy is a rather strong assumption and should be mentioned in the paper.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}