Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Authors: Xiaoli Tang
Han Yu
Zengxiang Li
Xiaoxiao Li
Venue: ICML 2025 poster (2025)
Keywords: Auction-based Federated Learning
bidding strategy
Primary area: general_machine_learning

Abstract: Auction-based Federated Learning (AFL) has emerged as an important research field in recent years. The prevailing strategies for FL data consumers (DCs) assume that the entire team of the required data owners (DOs) for an FL task must be assembled before training can commence. In practice, a DC can trigger the FL training process multiple times. DOs can thus be gradually recruited over multiple FL model training sessions. Existing bidding strategies for AFL DCs are not designed to handle such scenarios. Therefore, the problem of multi-session AFL remains open. To address this problem, we propose the Multi-session Budget Optimization Strategy for forward Auction-based Federated Learning (MBOS-AFL). Based on hierarchical reinforcement learning, MBOS-AFL jointly optimizes intersession budget pacing and intra-session bidding for AFL DCs, with the objective of maximizing the total utility. Extensive experiments on six benchmark datasets show that it significantly outperforms seven state-of-the-art approaches. On average, MBOS-AFL achieves 12.28% higher utility, 14.52% more data acquired through auctions for a given budget, and 1.23% higher test accuracy achieved by the resulting FL model compared to the best baseline. To the best of our knowledge, it is the first budget optimization decision support method with budget pacing capability designed for DCs in multi-session forward AFL.

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Xiaoli Tang'
Han Yu
Zengxiang Li
Xiaoxiao Li
Abstract
Auctieness oricuted probiem
T
Model uedt-aeeeedd prablem
problen DATA
Reszarvo Addnng Price
AUCTIONEER Ovennights
Traisier Tick
ESER
Auction-bart basied Federated Learning (AFL) has emerged as an important research field recent years. The prevailing strategies for FL data con- sumers (DCs) assume that the entire team of the required data owners (DOs) for an FL task must be assembled before training can commence. In practice, a DC can trigger the FL training pro- cess multiple times. DOs can thus be gradu- ally recruited over multiple FL model training sessions. Existing bidding strategies for AFL DCs are not designed to handle such scenarios. Therefore, the problem of multi-session AFL re- mains open. To address this problem, we propose the Multi-session Budget Optimization Strategy for forward Auction-based Federated Learning (MBOS-AF)) Based on hierarchical reinforce- ment learning, -I3OS-AFL.ointt optimizes inter- session budget pacing and intra-session bidding for AFL DCs, with the objective of maximizing the total utility. Extensive experiments on six benchmark datasets show that it significantly out- performs seven state-of-the-ar approaches. On average, MBOS-AFL achieves 12.28% higher util- ity, 14.52% more data acquired through auctions for a given budget, and 1.23% higher test accuracy achieved by the resulting FL model compared to the best baseline. To the best of our knowledge, it is the first budget optimization decision support method with budget pacing capability designed for DCs in multi-session forward AFL.
Protece
AFL Eceayston
PORKS
Figure I. An overview of auction- bastion-based federated learning (AFL).
1. Introduction
Federated Learning (FL) (Fan et al., 2025; Rendle, 2012;
Yang. 2020; Meng et al., 2024) has gained great attention for its ability to safeguard data privacy and user confidentiality in both academia (Qi al., 2025b;a; Tang et al., 2024c) and industry (Sun et al., 2024). Traditional FL approaches often assume that data owners (DOs), also known as FL clients, are willing participants in FL tasks, assisting data consumers (DCs), or FL servers, in training models. How- ever, this assumption may not always hold in practice, as DOs often weigh their participation against self-interest and cost-benefit considerations. To address this challenge, auction-based Federated Learning (AFL) has emerged as promising solution (Jiao al., 2019; Deng et al., 2021; Zhang et al., 2021; He et al., 2024).
As shown in Fig. 1, the key participants in AFL. are the auc- tioneer, DOs, and DCs. The data trading process between DOs and DCs modeled an auction, coordinated by the auctioneer. The auctioneer facilitates the flow of asking prices from DOs and bid prices from DCs. After receiving the bids, the auctioneer consolidates the results, notifies participants of the match-making outcomes, and determines the auction winners. Through these auction processes, DCs recruit DOs for FL training tasks. Once FL teams (i.e., DOs recruited for the FL training tasks) are formed, DCs proceed with the FL model training following standard FL protocols.
'College of Computing and Data Science, Nanyang Tech- nological University, Singapore 'Department of Electrical and Computer Engineering, The University of British Columbia, Canada Veector Institute, Canada. Correspondence to: Han Yu <han.yu@ntu ett..cuuuss...........
AFL methods can be divided into three categories (Tang et al., 2024a; Tang & Yu, 2023c): 1) data owner-oriented (DO-oriented), 2) cectionner--riennter and 3) data consumer- oriented (DC-oriented). DO-oriented AFL methods focus on helping DOs determine the amount of resources to com-
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
mit to FL tasks, and set their respective reserve prices for profit maximization. Auctioneer-orienteed AFL methods in- vestigate how to optimally match DOs with DCs as well as provide the necessary governance oversight to ensure desir- able operational objectives can be achieved (e.g. fairness, social cost minimization (Zhang et al., 2023)). DC-oriented AFL methods examine how to help DCs select which DOs to bid and for how much, in order to optimize key perfor- mance indicators (KPIs) within budget constraints, possibly in competition with other DCs.
based FL.. Extensive experiments on six benchmark datasets show that it significantly outperforms seven state-of-the- art approaches. On average, higher utility, 14.52% more data acquired through auctions for a given budget, and 1.23% higher test accuracy achieved by the resulting FL model compared to the best baseline.
achieves 12.28%
2. Related Work
Existing methods for DC-oniented issues can be further divided into two subcategories (Tang, 2024; Tang & Yu, 2025): i) reverse auction-bais basodd methods, and ii) forward auction- based methods.
This paper focuses on DC-oriented AFL, helping DCs bid for DOs. The prevailing methods in this domain require that the budget of a DC shall be maximally spent to recruit the entire team of necessary DOs before FI. model training can commence (Tang& & Yu, 2023b; Tang et al., 2024b; Tang & Yu, 2023a). In practice, throughout the FL model training process, a DC can recruit DOs over multiple training ses- sions. This is especially useful in continual FL (Yoon et al., 2021; Pan et al., 2016) settings where DOs' local data are continuously updated over time. Existing AFL approaches designed to optimize KPIs within single auctioning session cannot be directly applied in multi-session AFL scenarios, especially in scenarios with multiple DCs competing to bid for DOs from common pool of candidates. This is pri- marily due to the limitation that they are unable to perform budget pacing, which pertains to the strategic dispersion of a limited overall budget across multiple AFL sessions to achieve optimal KPIS over a given time frame.
Reverse Auction-based Methods: Developed primarily for monopoly AFL markets where there is only one DC facing multiple DOs, reverse auction-based methods like (Deng et al., 2021: Zhang al., 2021; Jiao et al., 2020; Zeng et al., 2020; Tang & Yu, 2024c; Le et al., 2020; Thi Le et al., 2021) address the challenge of DO selection through reverse auc- tions. The key idea of these methods is to optimally resolve the DO selection problem, targeting the maximization of KPIs specific to the target DC. Particularly relevant in sce- narios where disparate DOs vie for the attention of sole DC, these methods have progressed by integrating diverse mechanisms such as graph neural networks, blockchains, and reputation assessment.
Forward Auction- Methods: These methods are designed for situations where multiple DCs compete for the same pool of DOs (Tang & Yu, 2023b; 2024a; 2023a; 2024b; Tang et al., 2024b). The key idea of these methods lies in determining the optimal bidding strategy for DCs. The goal is to maximize model-specific key performance indicators. A notable example is Fed-Bidder (Tang & Yu, 2023b) which assists DCs to determine their bids for DOs. It leverages a wealth of auction-related insights, encompass- ing aspects like DOs' data distributions and suitability to the task, DCs' success probabilities in ongoing auctions and budget constraints. However, this method ignores the complex relationships among DCs, which are both competi- tive and cooperative. To deal with this issue, (Tang & Yu, 2023a) models the AFL ecosystem as a multi-agent system to steer DCs to bid strategically toward an equilibrium with desirable overall system characteristics.
To address this issue, we propose the Multi-session Budget Optimization Strategy for forward Auction-based Federated Learning (MBOS-A) AF... It is designed to empower DC with the ability to dynamically allocate its limited budget over multiple AFL DO recruitment sessions, and then opti- mize the distribution of budget for each session among DOs through effective bidding. The ultimate goal is to maximize the DC's winning utility. MBOS-AFE is grounded in Hierar- chical Reinforcement Learning (HRL) (Pateria et al., 2021) to effectively deal with the intricate decision landscape and the absence of readily available analytical remedies. Specif- ically, MBOS- AFL consists of two agents for each DC: 1) the Inter-Session Budget Pacing Agent (InterBP.) A), and 2) the Intra-Session Bidding Agent (IntraBMA). For each auctioning session, each DC's Int enterBPA opportunistically determines how much of the total budget shall be spent in this session based on jointly considering the quantity and quality of the currently available candidate DOs, as well as bidding outcomes from previous sessions. Then, the DC's Int r aBMA determines the bid price for each data resource offered by DOs in the AFL market within the session budget.
MBOS AFL falls into the forward auction-based methods category. Distinct from existing methods which focus on optimizing the objectives within a single auctioning session, it is designed to solve the problem of multi-session AFL budget optimization.
To our best knowledge, MBOS- AFL is the first budget op- timization decision support method with budget pacing ca- pability designed for DCs in multi-session forward auction-
2

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
3. Preliminaries
DC has exhausted its budget, it will forfeit future auc- tions. 4) Outcome Determination: Upon receiving bids from relevant DCs, the auctioneer determines the winning price based on an auction mechanism. It then compares the winning price with the reserve price set by each DO. If the winning price is lower than the reserve price, the auctioneer terminates the auction and informs the DO to initiate another auction for the same resources. Otherwise, the auctioneer informs the winning DC about the cost (i.e., the winning price) it needs to pay, informs the losing DCs, and informs the DO about the winning DC it shall join.
Generally, an AFL market consists of three types of participants (Tang et al., 2024a): 1) Data Owners (DOs): entities possessing potentially sensitive yet valuable data, who willing to share or sell access to their data resources for FL task training in exchange for appropriate compensation. 2) Data Consumers (DCs): organizations or individuals requiring data to train their machine learning models via FL. Auctionee::: a trusted third-party entity orchestrating the auction process between DOs and DCs. It facilitates the exchange of data resources for FL training tasks through auction mechanism, such the Second- Price Sealed-Bia (SPSB) auction (Yang et al., 2023).
AFL Market:
When the auctioning process for session s has been com- pleted or the DC has exhausted its budget, it initiates FL model training with the recruited DOs. Each DC pays the corresponding market prices the DOs it has recruited.
When a DO is ready to offer its services for FL task training, it notifies the auctioneer, specifying its bid request and the reserve price.' The auctioneer then announces the auction to all DCs currently participating in the AFL market. Any DC whose required the corresponding data resources aligns with the DO's offering submits a bid for the auction.
FL with Recruited DOs: After the auction- based DO re- cruitment process, the DC triggers the FL training process with the recruited DOs in session s, which is detailed in Appendix A.I.
Multi-Session Budget Constrained AFL Bidding: During the course of FL model training, a DC can initiate the FL training procedure (i.e., a training session) on multiple occa- sions, with the aim of recruiting DOs to improve model per- formance. Consider the scenario of multiple banks engaging in FL.. The dynamic nature of user data within these banks sets in motion perpetual cycle of updates, with continually refreshed data stored locally by each bank. As result, these banks systematically engage in repeated sessions of feder- ated model training periodically, during which the standard FL training protocol is followed. Let S denote the number of training sessions for the target DC, who has a budget B for all training sessions [S]. In each FL training session (s E [S], there are C. available qualified DOs, which can help train the FL model of the target DC. Each DO i E [C.] possesses a private dataset D; = ( //1
Let vg denote the reputation of DO i E [C.] (Shi & Yu, 2023) and æ E {0, 1} denote whether the target DC wins i. Then, the goal of the target DC across S sessions is to maximize the total utility of winning DOs² under the budget
B, which can be formulated as:
max
RE[S] EE(C.) F
Σ
Σ
x r
s.t.
Σ SE [E[C.]
x
x
Ps
S
B,
(1)
Following (Shi & Yu, 2023; Tang & Yu, 2024b), we calculate the reputation of each DO based on the GTGGSSapley method (Liu et al., 2022) technique and Beta Reputation System (BRS) (Josang & Ismail, 2002).
Data Owner Reputation Calculation:
We start by adopting the SV approach to calculate the contri- bution Φ; of each DO i during each training round towards the performance of the resulting FL model as
Following (Tang& & Yu, 2023b), we assume that each DO i become gradually available over time. Each DO i can trigger the following auction process: 1) Bid Request Initi- ation: DO i E [C.] generates a bid request about itself (c.g., identity, data quantity, etc.) and sends it along with the the reserve price (i.e., the lowest price it is willing to accept for selling the corresponding resources (Vincent, 1995)) to the auctioneer. 2) Bid Request Dissemination: The auctioneer disseminates the received bid request to the relevant DCs whose FL tasks are relevant to the data resources of the DO being auctioned. 3) Bidding Response: Each relevant DC evaluates the potential value and cost of the received bid request, and decides on a bid price based on its bidding strategy. The DCs submit their bids to the auctioneer. When
(wssta)) - s(ws) (W1
Φ =a
Σ SEAT(()
(2)
[8]
a is a constant. S represents the subset of DOs drawn from N. f(ws) denotes the performance of the FL model w when trained on data owned by S. The contributions made by the DOs can be divided into two types: positive contribution (i.e., O; 2 0); and 2) negative contribution (i.e., Φ; < 0). We use the variables pe; and ne; to record the number of positive contributions and the number of negative contributions made by each DO i, respectively.
FFolowing (Zhang et al., 2021; Tang & Yu, 2023b; Zhang et al., 2022a;b; Tang & Yu, 2023a), maximizing the total utility is equivalent to optimizing the performance of the global FL model obtained by the target DC.
FFllowing (Tang& & Yu, 2023b), we assume that DOs arrive and make their bid requests sequentially, one after the other.
3

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
formance, ultimately influencing the outcome across all training sessions. Moreover, this inter-session action serves as an initial state for the IntraBMA. It is worth noting that the InterBPA will stay static throughout given session 8. It is only updated when the session 8 is concluded. Fun- neling the inter-session action B, into the policy network of the Int ratMA BMA helps determine the intra-session actions, especially the initial intra-session action.
Intr vmniie Beddet Padiag Agrat
Honkin Aira
ba stuke Baddag Apust
Aatikencr 2
Aartkence A
batkeee
Hicrsted the Ovners
Reer sinil Dhis Diners
Fnteraed Tranteg
Fodersted
The primary function of the IntMA is to help DC bid for each DO i E [Ca] in session s in an efficient way, thus contributing to the crafting of the optimal budget allo- cation strategies under AFLS-AFL. The Int raBMA takes the dynamic DC state as the input, and produces the optimal action a¹ as the bid price for data owner i to be submitted to the auctioneer. As result, the IntraBMA will be up- dated upon every DO auction in session s. The synthesis of inter-session and intra-session actions culminates in the formulation of the DC's budget allocation strategy. In the following sections, we provide detailed descriptions of these two agents.
Figure 2. An overview of the proposed MB30S-AH approach.
Following BRS, the reputation value of of i can be computed as follows:
peN + 1 pa, + nc + 2"
e' ElBeta(pe, + no: + 1)] =
(3)
It is important to highlight that, as depicted in Eq. (3), the reputation of each DO undergoes dynamic updates as the FL model training process unfolds. Furthermore, in cases where there is no prior information available, the default initialization for the reputation value of is set to the uniform distribution, denoted as 8² = =N 1) = Beta(1, (1, 1).
4.1. Inter-session Budget Pacing Agent (InterBPA)
State: The state of the InterBPA A in session s E [S], de- noted as sx sttter, comprises two main segments. The first segment contains historical data derived from the preced- ing S sessions. These include the budgets allocated for each of the historical sessions, and the bidding outcomes of Intra in these sessions (including the bid prices for DOs, payment for DOs, and reputation of the recruited DOs), The second segment contains current session information (including the number of available DOs and the remaining budget). Thus, the formulation of inter is as follows:
The basies of Reinforcement Learning (RI.) could be found in Appendix A.2.
4. The Proposed MBOS- AFL Approach
Our primary objective is to help DCs recruit DOs across multiple sessions while adhering to budget constraints, with the overarching goal of maximizing the total utility. To ac- complish this, we must tackle two fundamental challenges: Determining the allocation of the to- tal budget B to given session s, B,; 2) Bidding Strategy: Determining the bid price by, for any given DO i in session s under the session budget B,, Since the AFL market is highly dynamic, it is difficult for DCs to obtain a closed-form ana- lytical solution for the above two problems. Therefore, we design MBOS-AFL based on RL (Sutton& Barto, 2018) to solve these problems without requiring prior knowledge. To determine the optimal budget allocation strategy and bidding strategy for DC to realize the objective outlined in Eq. (I), we design MBOS-AF based on HRL (Pateria et al., 2021).
s
1) Budget Allocation:
inter (b.-ss,.... "b,,,.,,,,,,...... bs.... P: P.-81,
B,.)
(4)
b, = (b)- (b1-1he((,,-))- P.-1 = 1p-_--ieec.--i1 and V--1 = 1i5-Ixe(C,_- The integration of historical con- text into the state design is pivotal, as it empowers the agent to understand the impact of its strategies on FL training over time.
Action: In session s, the action to be taken by the InterBPA A is to determine the budget allocated to the cur- rent session, inter which is expressed as:
It consists of two HRL-b budget allocation agents: 1)
Inter-session Budget Pacing Agent (InterBPA), and 2)
Intra-session Bidding Agent (IntraBMA). An overview of MBOS-AR is shown in Figure
inter a
B,,
(5)
Here, B, denotes the budget for session to bid for the data owners involved. This inter-session action plays a pivotal role in regulating the amount of budget to be disbursed by the DC during session thereby helping preserve the total budget B for potential future FL training sessions.
During each FL training session s, the InterBPA observes the current state within the model training environment. Sub- sequently, this observed state is channeled into the policy network of the InterBPA, generating the recommended inter-session action (i.e., setting the budget B, for session s). This action aims to enhance the current FL model per-
Reward: The inter-session reward for session 8, pinter is
4

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
determined by the average reputation of DOs recruited in s:
Algorithm 1 The training procedure of MBOS AFL
Initialize Qintre, Qinter with parameters gintra, g target networks of Qintre and Qinter with parameters gintre and ginter; replay memories Di and Dinter; target networks' update fre- quency r.
Σ
Eiglo, 2t se((,)
(6)
a' E {0, 1} denotes if the DC wins the auction for DO i.
1:
for 8 E [S] do
2:
Observe state s. sinter;
Discount factor: As the goal of a DC is to maximize the total utility derived from the recruited DOs for a given total budget B regardless of time, the reward discount factor of InterBPA is set to 1.
3: Compute B3 according to e-greedy policy w.r.t Qinter;
4:
for E [C,] do
5:
Observe state $2,- sintra:
6:
Compute 62 according e-greeyy policy w.r.t Qintra,
7:
Submit by to the auctioneer;
8:
Obtain rewards vy and the payment pi:
4.2. Intra-session Budget Management Agent (IntraBMA)
9:
B2 4 B, ph;
10:
Store transition tuples in Dintrr;
11:
Sample random minibatch of samples from D: Annte
State: The state of the Int.rBMA in session 8 during auction for DO i, denoted as gintra, consists of: 1) C. i: the remaining DOs in session s. 2) B,: the remaining budget of session s, and 3) vz: the reputation of DO i:
12:
y
ra
+
Y max ainttra Qintro (85, 0g o
outraa' :ginemn);
13:
Update gintre by minimizing Emlly inttra inttre o Qintra (se,t ( .ass
Sntve = [C= i, B., B. S5,
14:
Gintra + -gintra every I' steps:
15:
end for
16:
Obtain rewards rinter and the total payment ps during
Action: The action, denoted as fietra to be taken by the a, IntraBMP in session 8 for DO i E [C.] is to determine the bid price for i, i.e. 6,.
session 8;
17:
B B E Extcc,)
18:
Store transition tuples in Dinter;
19:
Sample a random minibatch of m samples from D;
Reward: The intra- reward for session s following the bid for DO i is defined the utility obtained from which is formulated as:
20:
y inter = rrat max max,i siater' Qinter a ginter' ginter);;
21:
Update genter by minimizing Em Iw inter Qinter (stnter, aagaer ainter; : ginterry]]
22: ginter +- ginter every I steps;
intra = u.
(8)
23: end for
Discount factor: Similar to InterBPA, the discount factor for the IntraBMA is also set 1.
associated with the corresponding agent. Q(s, a(s,'';) is the predicted action-value function of the corresponding agent for its next state s' and all possible actions a'. This target net- work is used to stabilize the learning process by providing a fixed target during training, which is updated periodically (every I' steps) to match the current action-valus network. Algorithm 1 is the training procedure for MBOS-AFI.
4.3. Training Procedure for InterBPA and IntrABMA
Following (Tang & Yu, 2023a), InterBPA and IntraBMA leverage the Deep Q-Network (DQN) tech- nique (Mnih al., 2015). Both agents use deep neural net- works (DNNs) to model the action-value function Q(s,, a), parameterized by ginter and (intra, respectively. To im- prove stability during training, we pair these networks with a similar DNN architecture parameterized by ginter and gintre, respectively (referred to as the target networks), which also approximates Q(s, To update ginter and gintre, the training conducted by minimizing the follow- ing loss function: L(0)= ! [=saan"")))D Q(s,, a; {) The replay buffer, D, is a storage mechanism for transition tuples {(s, a, r, r'), } where s is the new observation fol- lowing action a based on the state s, resulting in reward r. This buffer allows the agent to learn from its past experi- ences by randomly sampling batches of transitions during training. y represents the temporal difference target, and is computed as y = = max. Q(+, a'; 0). 7 is the discount factor, i represents the parameters of the target network
5. Experimental Evaluation
5.1. Experiment Settings
Dataset:
The performance assessment of MBOS-AFI. is conducted on the following six widely-adopted datasets in FL studies: 1) MNISTS, 2) CIFAR-10*, 3) Fashion-
MNIST (i.c., FMNIST) (Xiao et al., 2017). 4) EMNIST- digits (i.e., EMNISTD). 5) EMNIST-letters (i.e., EMNISTL) (Cohen et al., 2017) and 6) Kuzushij-MNIIST (i.e., KM- NIST) (Clanuwat et al., 2018). The Fl. models used are the same as those employed in (Tang & Yu, 2023b).
hhtps://www.cs.cormm/ http:///wwwww..ooooooooooooooo// wwt.s//w.oooooo.cccct.cc//aaaaa.
5

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Comparison Approaches: We evaluate the performance of MROOS-AFF against the following seven AFL bidding ap- proaches in our experiments: Constant Bid (Const) (Zhang et alll,20111 2014), Randomly Generated Bid (Rand) (Zhang et al., 2021; 2022b), Below Max Utility Bid (Bmub), Linear- Form Bid (Lin) (Perlich et al., 2012), Bidding Machine (BM) (Ren et al., 2017), Reinforcement Learning-based Bid (RLB) (Tang & Yu, 2023a; 2024b), FedBidder-sim (FBs), and Fed-Bidder-com (FBe) (Tang & Yu, 2023b). Details can be found in (Tang et al., 2024b).
shown in Table 1 with MBOS- -AFL improving the test accu- racy by 1.23% on average.
In addition, the comparative results under the Non-IID data with noise scenario can be found in Table 2. It can be observed that under these two different settings, the pro- posed method MBOS AFL consistently outperforms exist- ing methods in terms of achieving higher FL model accuracy. In particular, on average, MBOS- LFF achieves 1.49% and 1.72% higher FL model accuracy compared to the best per- formance achieved by baselines under the 10% noisy data and 25% noisy data settings, respectively. All these results demonstrate the effectiveness of our approach in helping DCs optimize their budget pacing and bidding strategies for DOs under the emerging multi-session AFI, scenarios.
Experiment Scenarios: We compare MBOS-A4 -FL with baselines under two main experiment scenarios with each containing 10,000 DOs: 1) IID data, varying dataset sizes, without noise: In this scenario, the sizes of datasets owned by various DOs are randomly generated, ranging from 500 to 5,000 samples. Additionally, all the data are independent and identically distributed (IID), with no noise. 2) Non- IID data, with noise: In this experimental scenario, we deliberately introduce data heterogeneity by adjusting the class distribution among individual DOs. Following (Tang et al., 2024b), we implement the following Non-III) setup. We designate class (on datasets other than EMNISTL) or 6 classes (on EMNISTL) as the minority class and assign this minority class to 100 DOs. As result, these 100 DOs possess images for all classes, while all other DOs exclu- sively have images for the remaining nine classes, excluding the minority class. In this experiment scenario, each DO holds 3,000 images. Additionally, we simulate scenarios in which the minority DOs contain 10% or 25% noisy data. The implementation details can be found in Appendix A.3.
Lin and Bmub typically outperform Const and Rand due to the use of utility in the bidding process. However, Bmub is less effective than Lin due to the reliance on randomness. Meanwhile, the more advanced methods BM, FBs, FBc, RLB and MBOS-AFL perform significantly better than the simpler approaches. This is largely due to the inclusion of auction records (including auction history and bidding records) and the use of advanced learning methods.
RLB and MBOS-ARE AFL both outperform BM, FBs, and FBc, due to their ability of adaptive adjustment to the highly dy- namic auction environment. While BM does consider mar- ket price distribution, it derives this distribution by learning the prediction of each bid request's 's market price density, which may lead to overfitting. In contrast, FBs and FBc obtain the market price distribution via a predefined winning function, which helps predict the expected bid costs more accurately. However, BM, FBs and FB are still static bid- ding strategies. They are essentially represented by linear or non-linear functions whose parameters are derived from historical auction data using heuristic techniques. Subse- quently, these parameters are applied to new auctions, even if the dynamics of these new auctions may vary significantly from those in the historical data. The inherent dynamism of the AFL market poses a considerable challenge for these static bidding methods, making it hard for them to consis- tently achieve desired outcomes in subsequent auctions.
Evaluation Metrics: To evaluate the effectiveness of all the comparison methods, we adopt the following three metrics: 1) the number of data samples won by the DC (#data), 2) the utility obtained by the DC (utility), and 3) the test accuracy (Ace). More details could be found in Appendix A.4.
5.2. Results and Discussion
To conduct a comparative analysis of bidding strategies based on these metrics, we carry out experiments across six datasets, each with varying budget settings. These settings span the range of (100, 200, 400, 600, 800). The results are shown in Tables 1, 2, and Figure 3.
While RLB optimize its bidding process with dynamic pro- gramming, is susceptible to the drawback of immediate re- wards, which might result in indiscriminate bidding for data samples without considering their associated costs. This issue is effectively addressed by MBOS-AFI. Moreover, it is worth highlighting that RLB is not designed for optimiz- ing budget allocation across multiple sessions. This is a distinction where
Table shows the results of various comparison methods under the IID data, different sizes of DOs datasets without noisy samples scenario. It can be observed that under all six datasets and five budget settings, MBOS-AFL. consistently outperforms all baseline methods in terms of both evaluation metrics. Specifically, compared to the best- perforrnng base- line, PBOSS-FFL achieves 12.28% and 14.52% improve- ment in terms of total utility and the number of data samples won, respectively. Figure 3 shows the corresponding test accuracy. The results align with the auction performance
offers significant advantages.
The test accuracy achieved by the FL models trained under all bidding strategies on CIFAR-10 is consistently lower than that on other datasets. This can be attributed to the base
6

---

Multi-Sessioe Budget Optimization for Forward Auction- basion Federated Learning
Table/ 1. Comparison results under the scenario of IID data, different sizes of DOs datasets without noisy samples. The best results are
highlighted Bold. Ours represents MBOS-BE AFL.
MNIST
CIFAR
FMINSTT
EMNIST
EMNISTL
KMNIST
Badget
Mithool
Edira
utiliny
Adann
miility
Ialat
iillity
3data
unilry
Edasa
urility
Adata
millity
8,32
9.897
10,723
6.46
7,638
6.52
7.359
7.02
6.75
Rond
9,125
8.41
8,721
8.43
9,743
8.09
8,853
8.10
6,822
7.97
8,940
7.96
Bonub
9.246
9.03
11,302
9.19
12.274
8.76
10,382
8.91
6,485
9.15
10,551
8.62
Lin
9,461
10.28
11,426
10.17
13,523
9.84
10,673
10.33
8,220
10.51
10,594
9.97
100
BM
12.324
11.95
13,367
11.85
15.321
12.65
14,399
12.19
15.157
12.27
14.501
12.46
FRs
13,985
14.51
14,259
13.51
16.373
13.53
15.321
13:46
14,408
13.44
15,509
13.54
FBc
13,809
13.84
13,984
13.70
15,843
13.42
16,772
14.23
14.168
13.67
16,927
13.64
13,892
14,263
17.783
13.95
15,989
15,544
14.40
16,027
14.33
Ours
14,944
16.59
17.397
17:47
19,064
18.19
18,674
17.46
16.317
18.59
18,687
16.55
Const
11,057
12,043
8.52
13,876
10.33
9.31
Rond
10,895
10.06
11,894
10.00
14,898
9:90
12,452
10.34
12,808
10.42
12,601
10.05
Bonu
16.582
9.58
17,021
10.60
25,327
10.60
17,817
10.40
20,966
11.43
17,878
10.97
Lin
17,803
13.14
17,849
12.88
26,880
12.88
19,135
12.61
27,860
12.70
19.553
12.97
200
BM
23,584
14.97
20.836
15.11
31,945
15.92
21,656
15:03
35,016
15.29
21.722
15.70
27,813
17.70
28,456
17.61
34,936
17.09
26.994
17.01
31,743
17.40
27,087
17.49
FBc
28,000
17.51
29,835
17:24
36,873
17.5 17.58
27,863
16.60
34,686
16.99
27,88
17.89
29.468
30.138
L7.52
35,548
17.04
26.748
37.122
17.82
25,819
17.23
Ours
33,045
21,99
35,163
21.08
39,982
23.72
35,856
19,59
37,645
22.43
35,737
18008
Const
14,395
8.34
X-41
Rand
13,195
9.86
16,372
9.71
17,844
6.87
17,003
7.13
6.431
9.02
17,051
9.20
Benub
23.378
10.90
25,631
11.16
31,487
10.86
24.756
10.05
23.639
10.63
24,869
11.33
Lin
24.523
11.58
26,830
14.13
32,677
14.24
25,669
14.28
36,261
14.31
25,802
11.16
400
RM
38,516
16.46
30,173
16.54
38,552
16.90
30,878
17.26
41,050
17.66
31,077
17.61
50,983
38,452
39,236
18.54
38.452
18.69
40,605
19.04
38,566
19:09
FBc
50,146
19.23
39,817
19.10
41,582
18.37
40,663
18,40
39,555
18.85
40,768
18.58
51.643
42,731
18.84
37.748
13.077
37,813
19.55
Ours
56,872
23.65
53,672
22.71
$2,386
23,00
47,135
19.32
46,341
23.83
47,262
19.75
17,895
19,378
21.391
19,832
10.596
9.55
19,982
8.92
Rand
19,903
8:68
20.184
9.07
20,853
11 11.69
18.838
10.37
24,581
9.15
18,966
9:83
Beub
30,164
29,174
11.85
29,660
12,06
33,768
11,94
29,
11:97
Lin
32,973
15.62
30,375
15.59
40,128
15.08
34,452
15.16
47.484
15.61
34,629
15.62
600
RM
49,8077
17.09
49,272
17:43
47,533
18.06
38,743
17.85
51.454
18:23
38,943
18:54
62,396
20.49
50,384
46,731
19.54
45.232
50,482
20.29
45,288
20:29
FBc
61,478
20.31
52,836
20.24
$2,843
19:92
48,767
19.38
49,468
20.04
48,958
20.05
63,672
58,273
50,172
19:26
59,155
20.53
20.14
Ours
66,654
21.72
60,737
22.82
63,
24.17
58,462
23.01
63,441
23.54
5H,522
21.72
23,047
21,753
26,311
11.13
17,875
22,705
11.30
Rand
24,853
14.09
22,845
13.34
22,734
13.68
20.474
13.60
26.563
13:57
20,642
13.26
Brub
36,703
12.99
35,777
12.70
401275
13.47
36.648
12.91
38,570
13.08
36,732
13.17
Lin
39.651
16.79
38,561
16.88
47,823
16.55
40,537
16.67
59,390
16.86
40,727
16.76
800
BM
57.442
18.57
52.735
18.68
51,272
19.16
16,772
19.31
65.086
19.41
46,933
19.59
FBs
70,496
22.09
62,842
22.07
54,453
21 077
51,863
21,02
67.470
21.54
51,942
21.69
FRc
72,845
22.04
63,112
22.00 06
15, 388
18
56.991
21.09
61,598
21.57
57,152
21.53
22.31
66.843
$2,621
20.92
68.943
21.78
$7,900
21.92
Ours
77,821
22.40
71,244
23.46
64,739
23.12
62,579
22.57
70,393
23,04
59,711
22.18
Const
Rand
Bmub
Lin
11911 BM
FBs
FBc
RLB
MBOS-AFL
MNIST
CIFAR
FMNSST
48
80
87-
46-
78
86
44-
76
ACC150)
ACC
88
42
84
40
72
38
70-
36
100
200
400
600
800
100
200
400
600
600
100
200
400
600
800
Budget EMNISTD
Budget EMNISTL
Budiget
KNMSS
85
80
76
84
74
83
ACC//9
72
ACC/99)
20-
70-
88
79
78
661
100
200
400 Budget
600
800
160
200
460
600
860
160
200
400 Budget
600
800
Budget
Figure 3. Comparison of accuracy under the scenario of IID data, different sizes of DOs datascts without noisy samples.
7

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Table Comparison of accuracy under the Non-IID data with noise scenario. 10% and 25% represents 10% and 25% noisy data,
respectively. Bud. represent budget and Ours represents MBOS AFL.
MNIST
CIFAR
FMNIST
EMNIST
EMNISTI.
KMNIST
Bod.
Method
10%
25%
10%
25%
10%
25%
10%
25%
10%
25%
10%
25%
Canst
70.03
12.88
13.97
61.45
57.87
77.02
76.46
64.92
58.21
59.63
Rand
69.61
65.42
10.57
10.83
62.70
59.48
78.69
77.97
63.97
62.83
57,01
59.12
Baub
71.22
70.61
15.37
12.94
63.32
60.45
78.42
77.37
66.88
65.19
61.83
61.76
Linn
72.36
70.32
18.65
17.41
64.04
64.13
78.62
77.44
66.47
64.07
62.72
62.97
100
BM
72.31
71.65
19.50
19.62
67.35
66.25
79.50
78.42
67.17
64.62
64.55
63.77
FBs
73.23
72.32
23.59
22.03
70.97
70.26
79.51
78.35
68.35
65.94
65,82
64.35
FBc
7
74.80
23.42
22.26
71.29
70.68
79.92
78.93
64.78
65.47
63.88
RLB
73.07
73.11
22.94
22.98
71.03
69.55
79.83
78.66
68.20
65.57
65,38
63.93
Ours
73.79
75.22
23.88
23.24
72.31
71.42
80066
79.29
69.26
66.76
66.15
65.08
Coust
70.73
66.38
10.08
11.08
63.74
60.T6
77.98
77.52
67.84
06.16
58.44
58,29
Rand
69.48
68.96
10.32
10.26
63.86
59.63
78.63
78.19
68.24
66.88
59.25
58.09
Baub
71.81
70.52
13.39
13.03
63.83
62.18
79.37
78.37
69.09
67.42
63.04
63.34
Lin
72.98
70.55
19.07
17.96
64.43
64.16
79.43
78.43
69.96
68.44
67.07
66.09
200
RM
73.43
72.48
20.36
20. 14
64.53
70.01
80.52
79.40
70.19
67.35
69.01
67.63
FBs
74.69
72.17
23.82
22.79
71.49
71.99
80.28
79.27
69.65
67.57
69.77
68.69
FBc
74.29
72.99
23.61
22.58
71.86
71.61
80.37
79.52
70.70
68.45
68.75
67.04
RLB
74.33
73.26
23.77
23.14
71.52
70.74
80.48
79.52
70.13
68.11
70.52
70.48
Ours
75.60
75.72
24.94
24.52
72.98
73.13
$1.31
66.10
71.39
69:05
71.13
71.27
Const
71.06
68.34
17.09
16.96
64.01
58.93
78.49
77.98
68.19
66.69
68.66
68.33
Rand
67.74
20.90
60.58
78.62
78.43
70.36
69.75
Baub
72.27
70.26
22.21
20.49
64.37
63.15
79.97
78,90
69.71
68.11
69.93
68.56
Lin
72.99
71.02
24, 18
22.94
65.52
65 44
80.01
78.99
70.53
69. 12
70.37
69. 10
BM
73.01
25.59
23.74
65.87
68.38
80,90
79.91
71.62
70.35
71.58
70.44
FBs
75.85
73.53
26.47
24.50
71.72
70.06
81,36
80,22
71.75
70.17
71.93
70.85
FBe
75.66
73:77
26.21
24.27
72.03
71.95
81.29
80.18
71. 88
70.38
71.01
69. .6.
RLB
75.25
74.96
26.78
24.83
72.31
72.24
81.55
80.47
71.99
70.59
72.45
70.72
Ours
76,59
76.33
27.65
25,86
73,85
73,63
81,86
80.69
72.54
71.84
73,38
71.66
Const
71.05
69.36
23.10
21.666
64.66
61.77
79.28
78.49
68.39
67.01
69.21
68.69
Rand
68.79
69.05
22.72
20.32
64.39
62.49
79.25
78.83
69.31
67.95
70.19
69.74
Bmub
71.95
71.07
18.90
22.02
64.41
63.78
80.68
79.38
70.49
68.71
70.78
69.
Lin
73.54
72.57
24.43
24.79
66.92
18
80.86
79.58
71.44
69.92
71.21
69.94
600
BM
75.25
73.58
28.30
26.62
67.21
67.80
81.42
80.26
72.47
71.07
71.97
70.82
FBs
76.18
76.16
28.85
27.25
73:55
71.81
81,47
80.34
72.51
71.06
72.26
72.23
FBce
76.25
73.98
29.07
28.95
74.14
73.31
81.49
80.31
72.51
70.99
72.
72.84
RLB
76.06
73.15
28.52
29.60
73.85
73.05
81.68
80.60
73.07
71.64
73.41
72.81
Ours
76.93
76.71
29,91
74.46
74,05
82.16
73.21
71,86
73.79
Const
67.21
66.43
23.63
21.99
67.17
64.97
79.64
78.81
68. 85
67.49
69.449
69.01
Rand
68.95
71.02
24.54
20.66
68.15
65.32
79.78
79.23
70.13
68.75
70.91
70.11
Baub
72.16
25.97
19.45
69.24
66.51
79.77
70.80
71.52
70.60
Lin
75.11
72.66
25.46
28.06
71.87
69:03
81.37
80,12
71.61
16.16
71.76
70.46
800
BM
75.28
73.89
28.76
29.00
72.83
70.31
81.64
80.58
72.89
71.63
73.09
71.74
FBBs
75.04
29.54
30.18
75.92
73.86
80.83
72.99
71.72
73.42
72.20
RLB
76.31
76.34
30.05
30.81
76.39
74.72
82.06
81.07
73.62
72.37
74.90
73.18
Curs
77.29
76.78
82.82
32.46
77.10
75.37
82.47
82.69
73.77
73.35
75.39
73.82
model adopted for FL training. As mentioned in Section 5.1, the accuracy reported in these two figures is with re- gard to the VGG11 network. Nevertheless, even with such a less effective base model, MBOS still significantly outperforms other baselines.
knowledge, it is the first budget optimization decision sup- port method with budget pacing capability designed for DCs in multi-session forward auction-based FL..
A Akknowledgements
6. Conclusions
This research is supported, in part, by the Ministry of Edu- cation, Singapore, under its Academic Research Fund Tier I (RG101/24); the RIE2025 Industry Alignment Fund Indus- try Collaboration Projects (IAF-ICP) (Award 12301E0026), administered by **STAR, as well as supported by Alibaba Group and NTU Singapore through Alibaba-NTU Global e-Sustainability CorpL.ab (ANGEL); and the National Re- search Foundation, Singapore and DSO National Laborato- ries under the AI Singapore Programme (AISG Award No. AISG2-RPP2020-0199 Xiaoxiao Li is supported by CIFAR AI Chair Awards and Canada Research Chair Fellowship.
In this paper, we propose the Multi-session Budget Optimization Strategy for forward Auction-based FL (MBOS-AFL). It is designed to empower FL DCs with the ability to strategically allocate budgets over multiple FL training sessions and judiciously distribute the budget among DOs within each session by bidding with different bid prices, in order to maximize total utility. Based the hierarchical reinforcement learning, MBOS-AFI, jointly optimizes inter-session budget pacing and intra-session bid- ding for DCs in the AFL ecosystem. To the best of our
8

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Impact Statement
Meng, L., Qi, Z., Wu, L., Du, X., Li, Z., Cui, L., and Meng, X. Improving global generalization and local personalization for federated learning. IEEE Transactions on Neural Networks and Learning Systems, 36, 2024.
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be
specifically highlighted here.
Mnih, V. et al. Human-level control through deep reinforce- ment learning. Nature, 518(7540):529 533, 2015.
References
Pan, Z.,, Yu, H., Miao, C., and Leung. C. Efficient collabo- rative crowdsourcing. AAAI, pp. 4248-4249, 2016.
Clanuwat, T., Bober-Irizar, M., Kitamoto, A., Lamb, A., Yamamoto, K., and Ha, D. Deep learning for classical japanese literature. arXiv preprint, pp. 1812.01718, 2018.
Pateria, S., Subagdja, B., hwee Tan, A., and Quek, C. Hier-
archical reinforcement learning: comprehensive survey. ACM Computing Surveys, 54(5): 10:)))99991110335 2021.
Cohen, G., Afshar, S., Tapson, J., and Van Schaik, A. EMNIST: Extending MNIST to handwritten letters. In IJCNN, pp. 2921 2017.
Perlich, C., Dalessandro, B., Hook, R., Stitelman, O.,
Raeder, T., and Provost, F. Bid optimizing and inven- tory scoring in targeted online advertising. In KDD, pp. 804-812, 2012.
Deng, Y.. Lyu, F., Ren, J., Chen, Y.-C., Yang, P., Zhou, Y., and Zhang, Y. Fair: Quality-aware federated learning with precise user incentive and model aggregation. In INFOCOM, 2021.
Qi, Z., Meng, L., Li, Z., Hu, H., and Meng. X. Cross-silo fea- ture space alignment for federated learning on clients with imbalanced data. In The 39th Annual AAAI Conference on Artificial Intelligence (AAAI-25), pp. 19986-19994, 2025a.
Fan, T., Gu, H., Cao, X., Chan, C. S., Chen, Q. Chen, Y., Feng, Y., Gu, Y., Geng, J., Luo, B., et al. Ten chal- lenging problems in federated foundation models. IEEE Transactions on Knowledge and Data Engineering, 2025.
Qi, 7.,, Zhang, R., Meng, L.., Wu, W., Zhang, Y., and Meng, X. Global intervention and distillation for fed- erated dittr---iittibution generalization. arXiv preprint 2ll::250400883 2025b.
He, Y., Ma, L., and Zhang, W. Should the title of talents be necessary? study on the "open bidding for selecting the best candidates" mechanism. International Journal of Crowd Science, 8(1):49-55, 2024.
Ren, K., Zhang, W., Chang, K., Rong. Y., Yu, Y., and Wang. J. Bidding machine: Learning bid for directly optimiz- ing profits in display advertising. TKDE, 30(4))645-65) 2017.
Jiao, Y., Wang, P., Niyato, D., and Suankaewmanee, K. Auc- tion mechanisms in cloud/fog computing resource alloca- tion for public blockchain networks. IEEE Transactions on Parallel and Distributed Systems, 30(9):1975-1989, 2019.
Rendle, S. Factorization machines with libFM. TIST, 3(3): 57:1-57:22, 2012.
Jiao, Y., Wang, P., Niyato, D., Lin, B., and Kim, I. Toward an automated auction framework for wireless federated learning services market. IEEE Transactions on Mobile Computing, 20(10):3034 3048, 2020.
Robbins, H. and Monro, S. A stochastic approximation method. The annals of mathematical statistics, pp. 400- 407, 1951.
Josang, A. and Ismail, R. The beta reputation system. In Bled eConference, volume 5, pp. 1502-2511. Citeseer, 2002.
Shi, Y. and Yu, H. Fairness-aware client selection for feder- ated learning. In ICME, 2023.
Sun, H., Tang, X., Yang, C., Yu, Z., Wang, X., Ding, Q., Li, Z., and Yu, H. Hifi-gas: Hierarchical federated learning incentive mechanism enhanced gas usage estimation. In he 36th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24), pp. 22824-22832, 2024.
Le, H. T., Tran, H., Tun, Y. K., Han, Z., and Hong, C. Auction based incentive design for efficient federated learning in cellular wireless networks, In WCNC, pp. 1-6, 2020.
Liu, Z. et al. GTG-Shapley: Efficient and accurate partici- pant contribution evaluation in federated learning. TIST, 13(4):1-21, 2022.
Sutton, R. S. and Barto, A. G. Reinforcement learning; An introduction. MIT press, 2018
Tang, X. Stakcholldr-oriented decision support for auction- based federated learning. Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI'24), pp. 8515-8515, 2024.
McMahan, B. et al. Communication-efiiain learning of deep networks from decentralized data. In AISTATS, pp. 1273-1282, 2017.

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
Tang, X. and Yu, H. Competirive-ccooperatve multi-agent re- inforcement learning for auction-based federated learning. In Proceedings of the 32nd International Joint Confer- ence on Artificial Intelligence (IJCAI223, pp. 4262442264444 -220, 2023a.
Vincent, D. R. Bidding off the wall: Why reserve prices may be kept secret. Journal of Economic Theory, 65(2): 575-584, 1995.
Xiao, H., Rasul, K., and Vollgraf, R. Fashion-MNIST: novel image dataset for benchmarking machine learning algorithms. arXiv preprint, pp. 1708.07747, 2017. 2017.
Tang, X. and Yu, H. Uullttymaximizing bidding strategy for data consumers in auction-based federated learning. In Proceedings of the 2023 IEEE International Confer- ence on Multimedia and Expo (ICME"22 23).. pp. 330-335, 2023b.
Yang, S. task offloading solution for internet of vehi- cles using combination auction matching model based on mobile edge computing. IEEE Access, 8:53261-53273, 2020.
Tang, X. and Yu, H. Agent-oriented joint decision support for data owners in baaed federated learning. In Proceedings of the 2024 IEEE International Conference on Multimedia and Expo (ICME'24), 2023c.
Yang, Z., Liu, Q., Zhao, X., and Zhao, Y. Empirical evidence idea generation open innovation community. Inter- national Journal of Crowd Science, 7(1):40-45, 2023.
Yoon, J., Jeong, W., Lee, G., Yang, E., and Hwang, S. J. Federated continual learning with weighted inter-client transfer. In Proceedings of the th International Con- ference on Machine Learning (ICML' 21), 2021.
Tang, X. and Yu, H. A cost-aware utility- maximizing bid- ding strategy for auction-bais based federated learning. IEEE
Transactions on Neural Networks and Learning Systems, 2024a.
Zeng, R., Zhang, S., Wang, J., and Chu, X. Fmore: An incentive scheme of multi-dimensional auction for feder- ated learning in MEC. In ICDCS, pp. 278-288, 2020.
Tang, X. and Yu, H. Efficient large-scale personalizable bidding for multi-agent auction-basee federated learning. IEEE Internet of Things Journal, 2024b.
Zhang, J., WU, Y., and Pan, R. Incentive mechanism for horizontal federated learning based on reputation and reverse auction. In WWW, pp. 947-956, 2021.
Tang, X. and Yu, H. Fairness-aware reverse auction-basee federated learning. IEEE Internet of Things Journal, 2024c.
Zhang, J., Wu, Y., and Pan, R. Auction- based ex-post- payment incentive mechanism design for horizontal fed- erated learning with reputation and contribution measure- ment. arXiv preprint 2201:v220010011111 2022a.
Tang, X. and Yu, Reputation aware revenue allocation for auction-based federated learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 018832-20844 2025.
Zhang, J., Wa, Y., and Pan", R. Online nuction-based incen- tive mechanism design for horizontal federated learning with budget constraint. arXiv preprint, pp. 2201 10907,97 2022b.
Tang, X., Yu, H., Li, X., and Kraus, S. Intelligent agents for auction-basse based federated learning: A survey. In Pro- ceedings of the 33rd International Joint Conference on Artificial Intelligence (1JCAI'24, pp. 2253-821,, 2024a.
Zhang. J., Shu, Y., and Yu, H. Fairness in design: A framework for facilitating ethical artificial intelligence designs. International Journal of Crowd Science, 7(1): 32-39, 2023.
Tang, X., Yu, H., Li, Z., and Li, X. bias-free revenue- maximizing bidding strategy for data consumers in auction-basee federated learning. Proceedings of the 33rd International Joint Conference on Artificial Intelli- gence (IJCAI'24) 24), pp. 4991 4999, 2024b.
Zhang, W., Yuan, and Wang, J. Optimal real-time bidding for display advertising. In KDD, pp. 1077-1086, 2014.
Tang, X., Yu, H., Tang, R., Ren, C., Li, A., and Li, X. Dual calibration-basse personalised federated learning. In Pro- ceedings of the 33rd International Joint Conference 2024c.
Artificial Intelligence (4), pp.
Thi Le, T. H., Tran, N. H., Tun, Y. K., Nguyen, M. N. H., Pandey, S. R., Han, Z., and Hong, C. S. An incentive mechanism for federated learning in wireless cellular networks: An auction approach. IEEE Transactions on Wireless Communications, 20(8):4874 4887, 2021.
10

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
A. Appendix
A.1. Federated Learning with Recruited Data Owners
After the auction-based DO recruitment process, the DC triggers the FL training process with the recruited DOs in session s, which is detailed in Appendix A. Specifically, the FL process operates through communication between the recruited DOs and the target DC in round-by-round manner. In each training round & in session 8, the target DC broadcasts the current global model parameters w to the recruited DOs. Upon receiving w each DO i performs local update to obtain w based on its private data D;, guided by the objective function
argmin (,aa)),,LL((t) v)].
(9)
(-) represents the loss function, which depends on the FL. model aggregation algorithm and the current global model parameters w For instance, FedAvg (McMahan et al., 2017) calculates w by employing SGD (Robbins & Monro, 1951) for certain number of epochs using the cross-entropy loss. At the end of round I, DO sends its optimized parameters w to the target DC. The global model is then updated by aggregating these parameter updates from the DOs
D
w'/ = Σ
w
(10)
Di denotes the total number of data samples of all the recruited DOs in session
A.2. Reinforcement Learning Basies
A Markov Decision Process (MDP) mathematical framework for modeling decision-making in which an agent interacts
with environment through discrete time steps. MDP is formally defined by the tuple A, R, 7): 1) S represents the possible states in the environment, denoted as S. 2) A encompasses the feasible actions the agent can take. 3) P x A x 5 + [0, 1] is the transition probability function for the likelihood of transitioning between states when an action is taken, capturing environmental dynamics. 4) R S x A x S + R is the reward function, specifying immediate rewards upon state transitions due to specific actions, with the agent's 's aim to maximize cumulative rewards. 5) y E [0, 1] serves as the discount factor, reflecting the agent's preference for immediate rewards versus future rewards.
During the MDP process, the agent interacts with the environment across discrete time steps. At each time step, it selects an action AA based on policy π S + A, subsequently receiving a reward r, and the environment undergoes state transitions according to P.
The goal of MDP is to identify an optimal policy π : S - A that maximizes the expected sum of discounted rewards over time, given by max, E Σn 7 This entails finding the policy maximizing expected cumulative rewards. The value function V/ S + R is associated with each policy, quantifying expected cumulative rewards. The optimal value function V*: S > IR represents the maximum achievable expected cumulative reward achievable with the best policy from each state.
A.3. Implementation Details
In our experiments, we faced the challenge of not having a publicly available AFL bidding behaviour dataset. To address this issue, we track the behaviors of DCs over time during simulations to gradually accumulate data in four different settings. Each setting contains 160 DCs who adopted one of the eight bidding strategies listed in the Compared Approaches section.
In the first setting, each of the eight baseline bidding methods is adopted by one eighth of the DCs. In the second setting, as BM, Fed-Bidder variants (FBs and FBc) and RLB have AI techniques similar to MBOS-AFLE these four bidding strategies are adopted by three sixteenths of the total population, while the remaining four baselines are adopted by one sixteenth of the total population. In the third and fourth settings, as both Fed- Rid-Bidder variants and MBOS-AFL are designed specifically for AFL,, we set the percentage of DCs adopting FBs and FBc to be higher than those adopting the other six baselines. Specifically, under the third setting, 50 DCs adopt FBs and FBc, while 10 DCs adopt each of the other six baselines. Under the fourth setting, 65 DCs adopted FBs and FBc, while DCs adopted each of the other six baselines. We adopt the second-price sealed-bid (SPSB) auction mechanism in our experiments. By tracking the behaviors of DCs over time, we can gradually accumulate data in the absence of publicly available dataset related to AFL bidding behaviours.

---

Multi-Session Budget Optimization for Forward Auction-based Federated Learning
To evaluate the effectiveness of MBOS-AFI, we create nine DCs, each utilizing one of the aforementioned bidding approaches join the auction for each bid request (i.e., each DO) in each session 8. Following (Tang & Yu, 2023b), bid requests are delivered in chronological order. Upon receiving a bid request, each DC derives its bid price based on its adopted bidding strategy. Subsequently, the auctioneer gathers the bid prices, identifies the winner, and determines the market price using the SPSB auction mechanism. The winning DC pays the market price to the DO. The process concludes when there are no more bid requests or when the budget is depleted.
MBOS-AFL utilizes fully connected neural networks with three hidden layers each containing 64 nodes to generate bid prices for target DO on behalf of their respective DCs. The replay buffer 2 of both the InterBPA and the Int raBrA set to 5,000. During training. both agents explore the environment using c-greedy policy with annealing rate from 1.0 to 0.05. In updating both Qintra and Qinter, 64 tuples uniformly sampled from 'D are used for each training step, and the corresponding target networks are updated once every 20 steps. In our experiments, we use RMSprop with learning rate of 0.0005 to train all neural networks, and set the discount factor 7 to 1. addition, we have set the number of candidate DOs within each session to 200 (i.e., C, = 200). The communication round in each session is set at 100, while the local training epoch is set 30, All experiments were conducted five times, and the averaged results are reported.
A.4. Evaluation Metrics
To evaluate the effectiveness of all the comparison methods, we adopt the following three metrics: 1) The number of data samples won by the DC (#data) defined as the cumulative number of data samples owned by DOs recruited by the corresponding DC until the budget session limits are reached. 2) The utility obtained by the DC (utility) is defined the cumulative reputation of DOs recruited by the corresponding DC until the budget or session limits are reached. 3) The test accuracy (Acc) is determined as the accuracy of the final FL model for the respective DC, up to the point where either the budget or session limits are reached.
12


Reviews:
[{"review_id": "HI98F1FgAe", "reviewer": "Reviewer_1uYb", "summary": "The authors deal with the multi-session budget allocation problem for data consumers in auction-based federated learning and propose the MBOS-AFL by introducing a a hierarchical RL framework.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your thoughtful suggestions and valuable feedback.\n\n>1. The selected baselines are outdated to some extent. It is suggested to incorporate more recent ones like [1, 2] with the same setting to help position the proposed method. In addition, Can you compare your bidding strategy in auctions with that of [3], though it is not designed for FL settings, it seems quite relevant. Also, can you comment on peer-based mechanisms (e.g., [4],[5],[6] or the references cited therein) vs auction-based mechanisms?\n\nBoth [1] and [2] propose bidding strategies for DCs in the AFL marketplace, but with different objectives. While [1] focuses on revenue maximization, differing from the utility maximization in this paper, the formulations are distinct. [2], however, aims to maximize utility and maintain the AFL marketplace's health, using a reinforcement learning approach similar to the RLB baseline method in this paper. \n\nBoth [3] and this paper focus on designing bidding strategies. However, different from [3] which aims to reach an equilibrium between buyers and sellers by designing bidding strategies for both buyers and sellers, we focus on guiding DCs (buyers) to bid for DOs (sellers) by budget management strategies to maximize their KPIs within a given budget. This difference in overall design goals makes it unsuitable to compare [3] (similarly [4][5][6]) with the proposed approach.\n\n>2. Limited exploration of communication costs in FL training rounds.\n\nThe proposed MSBO-FL framework consists of two main parts: the multi-session budget optimization mechanism (i.e., DO recruitment process) and the FL model training (described in “Preliminaries”).\n\nIn the DO recruitment part, computational complexity primarily arises from the utility estimation function and hierachical reinforcement learning (HRL)-based budget allocation phase. The HRL-based budget allocation is with a time complexity of $O(|S|·d + |C_s|·d + m)$, where $|S|$ represents the state space size, $|C_s|$ denotes the number of available qualified DOs in session $s$, $d$ is the neural network dimension, and $m$ is the minibatch size used in the training procedure.\n\nFor the utility estimation, the computational demand largely stems from the Shapley value-based contribution evaluation. We employ the Beta Reputation System combined with Shapley Value technique, which has a computational complexity of $O(T·N·log N)$, where $T$ and $N$ are the number of training rounds and data owners, respectively. Efficient contribution evaluation remains a challenging issue in federated learning, where high time complexity is common in related problems.\n\nIn the FL model training part, we adopt the FedAvg method, which is widely used in FL research. To compare the computational complexity of different bidding methods for data consumers in AFL, we can focus on the DO recruitment part, as the FL model training generally relies on FedAvg across methods. This recruitment part typically involves utility estimation and other functions, with the distinguishing feature of our proposed MBOS-AFL method being the two-level HRL-based budget allocation that optimizes both inter-session budget pacing and intra-session bidding strategies.\n\n>Other Comments Or Suggestions\n\nThanks a lot for your correction. We have revised the manuscript following your suggestions. \n\n>Q1: How does MBOS-AFL perform in scenarios with competing DCs using similar strategies?\n\nIf all DCs adopt the same bidding strategy, the differences in their individual historical data could lead to variations in their utility estimation and their states. This variance might result in differences in their bidding behaviors (i.e., actions). Moreover, in practice, different DCs target diverse data owners due to their distinct training tasks. These factors influence their bids through specific parameters and model approaches in their bidding strategies. Nevertheless, it remains an intriguing area for future research to analyze the dynamics and potential equilibria that arise when all data consumers adopt the same strategy.\n\n>Q2: Could the method be extended to handle non-static budget constraints (e.g., dynamically changing total budgets)?\n\nMBOS-AFL framework be naturally extended to accommodate dynamically changing total budgets with minimal modifications. The current design already incorporates state representations that track remaining budgets and historical allocations, making it well-positioned to handle budget fluctuations.\n\n>Q3: How does MBOS-AFL handle abrupt changes in DO availability across sessions?\n\nThe changes in DO availability have limited impact on the operation of MBOS-AFL. This is because such changes directly influence the state observed by the bidding agents, which then adjust their actions based on the learned policy. As a result, MBOS-AFL can dynamically generate appropriate bids in response to variations in DO availability, maintaining robust and adaptive performance under changing conditions.", "claims_and_evidence": "claims are well-supported.", "methods_and_evaluation_criteria": "The method is novel and appropriate for multi-session AFL.", "theoretical_claims": "This paper focuses on problem and algorithmic innovation, and empirical validation", "experimental_designs_or_analyses": "The experimental setup is sound, with detailed descriptions of datasets, baselines, and training protocols.", "relation_to_broader_scientific_literature": "This work advances AFL by addressing the multi-session budget pacing gap while integrating reputation systems and hierarchical RL.", "essential_references_not_discussed": "No major omissions.", "other_strengths_and_weaknesses": "The research problem is important and the proposed approach is clear and technique soundness. However, \n\n1. The selected baselines are outdated to some extent. It is suggested to incorporate more recent ones like [1, 2] with the same setting to help position the proposed method. In addition, Can you compare your bidding strategy in auctions with that of [3], though it is not designed for FL settings, it seems quite relevant. Also, can you comment on peer-based mechanisms (e.g., [4],[5],[6] or the references cited therein) vs auction-based mechanisms?\n\n\n2. Limited exploration of communication costs in FL training rounds.\n\n[1] Tang X, Yu H, Li Z, et al. A bias-free revenue-Maximizing bidding strategy for data consumers in auction-based federated learning[C]//Proc. IJCAI. 2024.\n\n[2] Tang X, Yu H. Competitive-Cooperative Multi-Agent Reinforcement Learning for Auction-based Federated Learning[C]//IJCAI. 2023: 4262-4270.\n\n[3] Chandlekar, Sanjay, et al. \"Multi-unit Double Auctions: Equilibrium Analysis and Bidding Strategy using DDPG in Smart-grids.\" Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems. 2022.\n\n[4] Richardson, Adam, Aris Filos-Ratsikas, and Boi Faltings. \"Budget-bounded incentives for federated learning.\" Federated Learning: Privacy and Incentive (2020): 176-188.\n\n[5] Witkowski, Jens, and David C. Parkes. \"A robust bayesian truth serum for small populations.\" Twenty-Sixth AAAI Conference on Artificial Intelligence. 2012.\n\n[6] Radanovic, Goran, Boi Faltings, and Radu Jurca. \"Incentives for effort in crowdsourcing using the peer truth serum.\" ACM Transactions on Intelligent Systems and Technology (TIST) 7.4 (2016): 1-28.】", "other_comments_or_suggestions": "1. Several citations are missing publication years or have unclosed brackets, like \"Tang and Yu ()\"（Page 2, Section 3). \n\n2. Table 1’s formatting could be improved for readability", "questions_for_authors": "1: How does MBOS-AFL perform in scenarios with competing DCs using similar strategies?\n\n2: Could the method be extended to handle non-static budget constraints (e.g., dynamically changing total budgets)?\n\n  3: How does MBOS-AFL handle abrupt changes in DO availability across sessions?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2uEg3lvdKn", "reviewer": "Reviewer_wy2c", "summary": "This paper introduces MBOS-AFL, a hierarchical reinforcement learning-based strategy for multi-session budget optimization in forward auction-based federated learning (AFL). The key idea is to enable data consumers (DCs) to dynamically allocate budgets across multiple FL training sessions (via an inter-session pacing agent) and optimize per-session bidding (via an intra-session agent). The method aims to maximize utility, defined as the aggregated reputation of recruited data owners (DOs). Experiments on six FL benchmarks (MNIST, CIFAR-10, etc.) demonstrate that MBOS-AFL outperforms seven baselines. The author addressed my concerns and I kept my score for technical contribution perspectives ## update after rebuttal", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your encouraging and insightful feedback. Below, we provide point-by-point explanations to key questions raised in the comments.\n\n>W1): The system uses the SPSB auction mechanism, why is not bidding the value truthfully optimal? That is the main promise of this type of auction.\n\nIn the generalised second price auctions, auction theory [1] proved that bidding the true value of the data resources is optimal towards maximizing the accumulated utility. However, the authors in [2] demonstrated that truthful bidding might not be optimal under budget constraints and limited availability of data resources. In this sense, it is necessary to design bidding strategies for data consumers to automatically calculate bid prices for the available data resources in view of the budget limit, instead of bidding with the prices in the simplified linear form.\n\n>W2）How to make sure the utility estimation is accurate? \n\nAccurate utility estimation is crucial for the proposed. In practice, it is recommended to continuously monitor and update the utility estimation model as new bidding data becomes available, ensuring its adaptation to potential changes in the AFL environment.\n\n>W3) The experiments could be enhanced by including some auction-based datasets or data from AFL market.\n\nThanks a lot for your suggestion. Since publicly available data from the AFL market is not accessible, and existing auction-based datasets, such as those from online advertising, are not directly applicable to a federated learning setting, we have followed established methods [3,4] to collect and generate the datasets used in our experiments. We will also explore additional auction-based datasets that could potentially be adapted for a federated learning setting.\n\n>W4) How practical is the proposed framework for real-world use?\n\nIndeed, the proposed method can be adapted for use in most FL settings that include an incentive mechanism with data consumers competing to bid for data owners [5,6,7], particularly in auction-based FL scenarios. A practical real-world application is in gas usage estimation within the power generation and delivery industry [8].\n\n>1) In page 7 (Figure 3) \"MultiBOS-AFL\" -> \"MBOS-AFL\"; the accuracy plots could benefit from clearer legends\n>2) In page 8 (Table 2), missing the percentage sign\n\nThanks for your correction and suggestions. We have revised the typos in the manuscript and revised the legend of the accuracy plots to make it clear following your suggestions. \n\n-----\n\n[1] Vijay Krishna. Auction theory. Academic press, 2009.\n\n[2] Weinan Zhang et al. Optimal real-time bidding for display advertising. In KDD, 2014.\n\n[3] Yutao Jiao et al. Toward an automated auction framework for wireless federated learning services market. TMC, 2020.\n\n[4] Rongfei Zeng et al. Fmore: An incentive scheme of multi-dimensional auction for federated learning in MEC. In ICDCS, pages 278288, 2020.\n\n[5] Yutao Jiao et al. Toward an automated auction framework for wireless federated learning services market. TMC, 2020.\n\n[6] Rongfei Zeng et al. Fmore: An incentive scheme of multi-dimensional auction for federated learning in MEC. In ICDCS, pages 278288, 2020.\n\n[7] Palash Roy et al. Distributed task allocation in mobile device cloud exploiting federated learning and subjective logic. Journal of Systems Architecture, 113(2):doi:10.1016/j.sysarc.2020.101972, 2021\n\n[8] Hao Sun et al, \"HiFi-Gas: Hierarchical Federated Learning Incentive Mechanism Enhanced Gas Usage Estimation,\" in IAAI-24, 2024.", "claims_and_evidence": "The claims are convincingly supported by experiments across diverse settings, including IID/Non-IID data and noisy conditions. The consistent outperformance of MBOS-AFL over RLB and Fed-Bidder variants highlights its novelty in multi-session pacing.", "methods_and_evaluation_criteria": "The hierarchical RL design effectively decouples pacing and bidding decisions. Benchmark datasets and realistic noise/heterogeneity settings ensure relevance to real-world FL challenges.", "theoretical_claims": "No theoretical claims are made; the work is empirically driven.", "experimental_designs_or_analyses": "The experimental design is extensive with five budget settings and multiple experimental scenarios.", "relation_to_broader_scientific_literature": "This paper studies bidding strategy for data consumers in AFL, advancing bidding strategies beyond data privacy concerns.", "essential_references_not_discussed": "All relevant AFL and RL works appear to be cited.", "other_strengths_and_weaknesses": "Strength:\n\n1) The paper’s contents are well-organized.\n2) The proposed method is technique soundness.\n3) The paper provides insight analysis through its analysis and comparison experiments.\n4) The paper proposes a new benchmark dataset processing for auction-based federated learning.\n\nWeakness:\n\n1) The system uses the SPSB auction mechanism, why is not bidding the value truthfully optimal? That is the main promise of this type of auction.\n2）How to make sure the utility estimation is accurate? \n3) The experiments could be enhanced by including some auction-based datasets or data from AFL market.\n4) How practical is the proposed framework for real-world use?", "other_comments_or_suggestions": "1) In page 7 (Figure 3) \"MultiBOS-AFL\" seems shall be \"MBOS-AFL\"; the accuracy plots could benefit from clearer legends\n2) In page 8 (Table 2), missing the percentage sign", "questions_for_authors": "refer to the Weakness", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hvnQriOAqu", "reviewer": "Reviewer_KgPU", "summary": "This paper explores the design of bidding strategies for data consumers in auction-based federated learning (AFL), involving three key stakeholders: (1) data owners, who are willing to share their potentially sensitive data in exchange for appropriate compensation; (2) data consumers, who require data to train their federated learning models; and (3) a trusted third-party “auctioneer,” responsible for orchestrating auctions between data owners and consumers to optimize the latter’s key performance metrics. typically assume a single-session setting. However, in practice, data consumers often recruit data owners across multiple training sessions. This limitation in existing works leads to suboptimal performance, as they fail to account for budget allocation across sessions and the complex interplay between inter-session and intra-session dynamics. This paper presents a novel budget allocation strategy under multi-session FL sessions for data consumers in AFL. By tackling a critical issue for AFL sustainability through a theoretically grounded and empirically validated approach, this work makes a valuable contribution to the field", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful and encouraging feedback. Below, we provide detailed, point-by-point responses to the key questions raised in the comments.\n\n>W1. More motivation of why multi-session auction scenario is realistic seems to be helpful. \t\n\nPractical federated learning environments typically require sequential collaboration phases where the collective model evolves gradually through repeated information consolidation and parameter adjustments. Consider healthcare applications: medical facilities (serving as information contributors) supply clinical data incrementally across various timeframes, leading to progressive enhancement of the shared analytical framework.\n\nThe operational reality demonstrates that participating entities join the collaboration asynchronously, while available funding resources may vary throughout the project lifecycle. This dynamic creates an ideal application for multi-round bidding structures. Such an approach accurately reflects the continuous nature of knowledge development in these collaborative ecosystems, where both informational assets and processing capabilities are distributed through successive competitive allocation events.\n\n>W2. The paper would benefit from a clearer exposition of its assumptions regarding data owner  behaviors. While the authors describe the decision decision-making process of data consumers, it's unclear whether these are based on empirical observations, theoretical models, or simplifying assumptions. A more explicit statement of the behavioral assumptions underpinning the model would strengthen the paper's foundation and help readers better understand the scope and limitations of the proposed approach.\n\nThe method we've developed operates without assumptions regarding the other types of stakeholders' behaviors. Rather, it features inherent adaptability to accommodate diverse stakeholder actions across multiple contexts. Our outlined decision protocols maintain flexibility and remain independent of underlying models, enabling effective functioning regardless of participant motivations or strategic choices. This versatility represents a fundamental advantage of our approach, as it facilitates widespread implementation across various aggregated federated learning environments without limitations imposed by strict behavioral assumptions or predefined conduct expectations.\n\n>W3. It is suggested to discuss the technical challenges of designing proposed mechanisms.\n\t\t\nThe first technical challenge lies in optimizing the computational efficiency of utility estimation. Our methodology implements a framework combining Shapley-derived valuations with beta-based reputation tracking to quantify the utility a DC can gain from each DO. However, while Shapley-based methods are conceptually straightforward, they are computationally intensive. Additionally, another challenge is ensuring robust winning price modeling across diverse data owner distributions, as well as balancing the bidding strategy’s performance with budget constraints. Addressing these complexities is crucial for adapting our approach to practical AFL market environments, and we will add further details on these points in the revised paper.\n\n>Typo\n\nThank you for pointing that out. We have corrected the typo in the revised manuscript.\n\n> What bidding strategies did the other bidders use in the experiments? Were their approaches similar or distinct?\n\nAs detailed in the experiment section, we compared diverse advanced bidding strategies such as FBs, FBc, among others, by various bidders. This setting is designed to address concerns regarding the potential impact of competitors with advanced bidding strategies. However, if all bidders adopt the same bidding strategy, the differences in their individual historical data could lead to variations in their utility estimation and winning price models. This variance might result in differences in their bidding behaviors. Moreover, in practice, different bidders target diverse data owners due to their distinct training tasks. These factors influence their bids through specific parameters and model approaches in their bidding strategies. Nevertheless, it remains an intriguing area for future research to analyze the dynamics and potential equilibria that arise when all data consumers adopt the same strategy.", "claims_and_evidence": "Extensive experiments across diverse scenarios (IID/Non-IID data, noisy settings, varying budgets) strongly support the claims. The results hold consistently across key metrics (utility, data volume, accuracy), and comparisons with baselines (e.g., RLB, Fed-Bidder) confirm the advantage of hierarchical RL. Reproducibility is ensured with detailed implementation guidelines in the appendix.", "methods_and_evaluation_criteria": "The hierarchical RL framework well suit the inter-session budget allocation and intra-session budget allocation case, addressing both pacing and bidding dynamics. The datasets used in this paper are widely adopted in the filed of AFL and considering of realistic scenarios (e.g., noisy, Non-IID data) ensures thorough evaluation. Metrics (utility, accuracy) align with the DC’s goal of maximizing model performance under budget constraints", "theoretical_claims": "No issue found with regard to theoretical claims", "experimental_designs_or_analyses": "The experiments conducted are rigorous: five runs with averaged results, varied budgets, and multiple noise levels. In addition, the inclusion of both IID and Non-IID settings strengthens validity.", "relation_to_broader_scientific_literature": "The work extends DC-oriented AFL by addressing multi-session budget pacing, a gap in prior single-session methods (e.g., Fed-Bidder, RLB). It connects hierarchical RL [Ref 1] to AFL, offering a novel integration.\n[Ref 1]  Pateria, S., Subagdja, B., hwee Tan, A., and Quek, C. Hier archical reinforcement learning: A comprehensive survey. ACM Computing Surveys, 54(5):109:1–109:35, 2021", "essential_references_not_discussed": "No critical omissions found.", "other_strengths_and_weaknesses": "S1. The paper is mostly clearly written.\nS2. The paper offers analysis and hence justification for the proposed bidding method, which is technically sound.\nS3. The empirical results are promising against other baseline bidding strategies, although existing methods are limited because the literature is scarce.\n\nW1. More motivation of why multi-session auction scenario is realistic seems to be helpful. \nW2. The paper would benefit from a clearer exposition of its assumptions regarding data owner  behaviors. While the authors describe the decision decision-making process of data consumers, it's unclear whether these are based on empirical observations, theoretical models, or simplifying assumptions. A more explicit statement of the behavioral assumptions underpinning the model would strengthen the paper's foundation and help readers better understand the scope and limitations of the proposed approach.\nW3. It is suggested to discuss the technical challenges of designing proposed mechanisms.", "other_comments_or_suggestions": "Typo: Figure 3, MultiBOS-AFL -> MBOS-AFL?", "questions_for_authors": "What bidding strategies did the other bidders use in the experiments? Were their approaches similar or distinct?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper is well-organized with clear motivation and method. The proposed method is novel. Its claims are well supported by theoretical and experimental analysis. The suggested refinement would be justify how the proposed scenario is aligned to real applications, for example, using dataset from real applications, and provide a better justification.
Decision: Accept (poster)