Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning
Authors: Mengmeng Chen
Xiaohu Wu
Xiaoli Tang
Tiantian He
Yew-Soon Ong
QIQI LIU
Qicheng Lao
Han Yu
Venue: NeurIPS 2024 poster (2024)
Keywords: Federated learning
business sectors
competition
self-interest
Primary area: machine_learning_for_other_sciences_and_fields

Abstract: Federated learning (FL) is a machine learning paradigm that allows multiple FL participants (FL-PTs) to collaborate on training models without sharing private data. Due to data heterogeneity, negative transfer may occur in the FL training process. This necessitates FL-PT selection based on their data complementarity. In cross-silo FL, organizations that engage in business activities are key sources of FL-PTs. The resulting FL ecosystem has two features: (i) self-interest, and (ii) competition among FL-PTs. This requires the desirable FL-PT selection strategy to simultaneously mitigate the problems of free riders and conflicts of interest among competitors. To this end, we propose an optimal FL collaboration formation strategy -FedEgoists- which ensures that: (1) a FL-PT can benefit from FL if and only if it benefits the FL ecosystem, and (2) a FL-PT will not contribute to its competitors or their supporters. It provides an efficient clustering solution to group FL-PTs into coalitions, ensuring that within each coalition, FL-PTs share the same interest. We theoretically prove that the FL-PT coalitions formed are optimal since no coalitions can collaborate together to improve the utility of any of their members. Extensive experiments on widely adopted benchmark datasets demonstrate the effectiveness of FedEgoists compared to nine state-of-the-art baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.

Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning
Mengmeng Chen', Xiaohu Wu', Xiaoli Tang", Tiantian Hea.4, Yew-Soon Ong2.3,4
Qiqi Liu Qicheng Lao', ; Han Yu
Beijing University of Posts and Telecommunications, China
CCollege of Computing and Data Science, Nanyang Technological University, Singapore Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore *Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore Schooo of Engineering, Westlake University, China
{chemmengmeng03366 xiaohu. wu, qicheng. -bbptt edu. cn, ; han. yu)otu. edu. sg, he_tiantian@en aettccfrr..ssataa... a-star sg, Aaugiqi@westlkke edu. cn
Abstract
Federated learning (FL) is a machine learning paradigm that allows multiple FL participants (FL-PTs) to collaborate on training models without sharing private data. Due data heterogeneity, negative transfer may occur in the FL training process. This necessitates FL-PT selection based on their data complementarity. cross-silo FL, organizations that engage in business activities are key sources of FL-PTs. The resulting FL. ecosystem has two features: (i) self-interest, and (ii) competition among FI.-PTs. This requires the desirable FL-PT selection strategy to simultaneously mitigate the problems of free riders and conflicts of interest among competitors. To this end, we propose an optimal FL collaboration formation strategy -FedEgoists- which ensures that: (1) FL-PT can benefit from FL if and only it benefits the FL ecosystem, and (2) a FL-PT will not contribute to its competitors their supporters. provides an efficient clustering solution group FL-PTss into coalitions, ensuring that within each coalition, FL-PTs share the same interest. We theoretically prove that the FL-PT coalitions formed optimal since no coalitions can collaborate together to improve the utility of any of their members. Extensive experiments widely adopted benchmark datasets demonstrate the effectiveness of FedEgoists compared to nine state-of-the-art baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.
Introduction
Federated learning (FL) is promising paradigm of distributed machine learning (ML) as it does require sharing raw data between FL participants (FL-PTs), thereby upholding the privacy considerations 10
54 55 56]. In the popular Federated Averaging (FedAvg) framework, multiple FL-PT's train shared model locally with their own datasets, and upload their local model updates to a central server (CS), which then aggregates these model updates and distributes the resulting global model to each FL-PT [32]. There are two types of FL 20 cross-device FL, FL-PTs are end-user devices such as smartphones or IoT devices, and CS is the final owner of the trained model. cross-silo FL, FL-PTs are companies or organizations in private or public sectors and are the final owners/users of the trained model. while the CS has the authority to coordinate the FL training process.
12
*Corresponding author
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Every company is self-interest;
x
There exists competitions between companies.
Ensures the absence of the free riders;
voods the conflict of interest; Optimal coalition formation with the two constraints
A FL Manager
above.
Figure I: An overview of the main motivation and results of this paper.
We consider in this paper the scenario of cross-silo FL where organizations in the business sector that engage in business activities are key sources of FL-PPTs [16]. The business sector is part of the private sector made up by companies and includes business that operate for profit; it is key driver of technological innovation. The application of FL in the business sector has been studied in diverse domains, including digital banking, ridesharing, recommender systems, and Blectric Vehicle charging services [28] 29 39
47 (33. Regional banks have different user groups from their respective regions and are independent [54], while the banks in the same region can compete for users (31] Recently, FL platform, MELLODDY, has been developed for drug discovery. including multiple companies from business sectors 335. Generally, competition exists when there are multiple organizations that are in the same market area and hope to unlock the power of FL [16 45].
From FL-PT's perspective, the scenario under study has features: (i) self-interest and (ii) competition among FL-PTss. In business sectors, individuals are self-interested. This magnifies the free-riding problem where some FL-PT's benefit from the contributions of others without making any contribution to the FL ecosystem 200 (22). Competition signifies that there is a potential conflict of interest between some two FL-PTs. Thus, two principles can simultaneously be used to meet the individual's 's needs: (1) a FL-PT can benefit from the FL ecosystem if and only if it can benefit the FL ecosystem, and (2)= a FL-PT will not contribute its competitors as well as the allies of its competitors [42]. The first principle is proposed and formulated in this paper for the first time, while the second principle has previously been considered for other scenarios 42) From a FL manager's perspective, its priority is to regulate the FL. ecosystem to meet the requirements of its customers (i.e., FL.-PTs).
This paper studies cross- silo FL in business sectors where there are typically a limited number of FL-PTs (e.g., 2 to 100) [6 27]. We consider both self-interest and competition for the first time in literature, although it has been well recognized that the free riding problem can be especially serious when self-interested FL-PTs engage in business competition 17 4T Since competition and self-interest are considered. all FL-PTs are partitioned into disjoint groups/coalitions, each with the common interest. The competition and benefit relationships among FL-PTs can be described as graphs. We use theoretical tools from graph theory and propose an efficient solution FedEgoist that can well satisfy the two principles above. Meanwhile, FedEgoists can help FL-PTs achieve the best possible ML model performances, i.e., subject to the two principles, the coalitions that FedEgoists finds are optimal in the sense that one coalition cannot increase the utility of any its members by collaborating with any other coalitions. Extensive experiments over real-world datasets have demonstrated the effectiveness of the proposed solution compared to nine baseline methods, and its ability to establish efficient collaborative networks in cross-silo FL with FL-PTss that engage in business activities. The main motivation and results of this paper are illustrated in Figure T
2
Related Work
The existing FL works consider the following three lines of works separately and don't address the self-interest and competition features simultancously.

---

Competition. Collaboration between competing companies is an important research area in business studies IT. Recently, there has been a growing interest in the study of competition in cross-silo FL where a key application area is the business sector. There are two cases. In the first case, all FL.-PTs are assumed to offer the same service in the same market area and compete against each other. Several works attempt to reduce the potential side effect after FL.-PT's join a fully competitive FL ecosystem. Wu and Yu [50) aim to maintain a negligible change in market share [5] [2 and analyze the achievability of this objective. Tsoy and Konstantinov [45] and Huang et al. [6 study the profitability of FL-PTs, but are taken under different assumptions on the source of extra profit brought by FL. Specifically, Tsoy and Konstantinov fixed budget to multiple services from different markets, and (ii) if FL-PT owns a better model, the consumer will thus allocate more of its budget to consume its service due to the improved service quality. Huang et al. [6] consider duopoly business competition and assume that, if the model-related service be improved by FI., customers will have a willingness to pay more and FL.PPTs thus have opportunities to increase their profits. In the second case, not all FL.PPT's compete against each other, like one assumption in this paper, where some FL-PPTss come from different market areas. Tan et al. [42] study how to avoid the conflicts of interest among FL-PTs for the first time and give a heuristic to realize Principle 2. However, Tan et al. 442 don't 't consider the self-interest feature of cross-silo FL in the business sector and cannot guarantee the optimal collaboration among independent coalitions without competition. The collaboration relationships among FL-PT's result in collaboration graph Gu. In their heuristic, FL-PTs are considered one by one and g. is thus constructed gradually to guarantee that for each FL-PT, there no path that can be reachable to or from its competitors. In this paper, we apply the concepts of cliques and strongly connected components in graph theory to satisfy the self-interest and competition features and further optimize the proposed algorithm.
[45 assume that: (i) each consumer allocates
Free-riding, Karimireddy et al. [22 show that naive scheme can lead to catastrophic levels of free- riding where the benefits of data sharing are completely eroded. By contract theory, they introduce accuracy shaping based mechanisms prevent free-riding.
Coalitions. A coalition is a cooperative relationship formed among different individuals due to certain interests. Each agent is only concerned with the contributions of other agents within the same coalition [ Prior studies mainly focus on alleviating the side effect of data heterogeneity by allocating proper collaborators with data complementarity to each FL-PT. Donahue and Kleinberg [9] provide an analytical understanding of what partition of FL-PT's leads to core-stable coalition structure for mean estimation and linear regression. Cui et al. 3 propose heuristic algorithm to compute the core-stable coalition structure for general learning tasks. To some extent, the CS can dictate the collaboration relationships among PLS only if doing so can better address the concerns of FL-PT's such as fairness and performance 6
59 60). Chaudhury et al. 4 treat all FL-PTs as a grand coalition and optimize a common model such that there is no other coalition S of FT.-PTss that could significantly benefit more by training model only using the data of S. Sattler et al. [37 propose a novel federated multi-task learning framework where the geometric properties of the FL loss surface is leveraged to group FL-PTs into clusters with jointly trainable data distributions. Ding and Wang 7 partition a total of n FL-PTs into K groups, where K < r and the FL-PTs with similar contributors are assigned to the same group. Then, a common model is returned to each group of FL-PTss such that the performance of using K models for n FL-PTss is good approximation to the the performance of learning personalized models for n FL-PTs respectively. However, the above works don't 't consider the adversarial relationships among FL-PTs.
13
3 Model and Assumptions
3.1 Relationship Graphs
There is a set of FL-PTs denoted by V = {U1, u..... ). Each FL-PT 8; is uniquely equipped with its own local dataset D, A benefit graph Go is deployed to describe the potential collaboration advantages between any two FL-PTs. Together with the competing relationships represented by Ge among different FL-PTss, we pave the way for the formation of the contribution graph Gu. The benefit graph Gú a weighted directed graph with an edge set E, (i.e., G, = (V, Ex)) and is used to evaluate the data complementarity between FL-PTs. It is defined as follows. For any two FL.-PT's ej and u, if i can benefit from vy's 's data, then there a directed edge from vy to e; (i.e., (uy. v.) E Fb) and the weight of this edge is Wy.s > 0 where larger value of Wy.a signifies larger benefit to &, brought by
3

---

ey. In contrast, if cannot benefit from the data of Uj. then (0y, v/) & Eb and W/2 = 0. The benefit graph can be computed by the hypernetwork technique like [5 B3 42 The competing graph G. is an undirected graph with an edge set Fec. i.e., G. (V, F.). For any two FL.-PTss by and V/. if they compete against each other, then there is an undirected edge between i to bj (i... (v;, 11)) E Fe.) If they are independent of each other, then (e4, 2/y) & E... The CS is assumed to be trustable. In the real world, the CS may represent an impartial and authoritative third-party (e.g., the industry association) [5]. Then, FL-PTs can report their competitive relationships to the third-party in person and confidentiality agreements can be signed between the third-party and FL-PTs. Each FL-PT o will report its competitors CS, as it hopes that CS will correctly utilize this information to prevent its competitors from benefiting from its data. Thus, CS has the knowledge of ge. Although v may benefit from by's 's 1, > 0). CS has the authority to determine whether v; can actually utilize vy's local model update information (i.e., indirectly use 0y 's data) in the FL. training process or not. matrix where Fj.a {0, 1}: for two different FI.-PT's ey and By Zz is set to one if 1y will contribute to on (i.e., Y/ will utilize by 's local model update information) in the FL training process and Fz is set to zero otherwise. X' defines directed graph 9u (V, E.). called the data usage graph: (2y, Uz) E Ey if and only if + i, Jj = 1; then, vy is said to be a collaborator or contributor of ez.
Let X' (15,4) be
Finally, we introduce some concepts in graph theory. Let G = (V,E) denote an arbitrary graph whose node set is V and whose edge set is E. For any subset S C V, an (induced) subgraph ç (S) of the graph G is such that (i) the node set of G (S) is V and (ii) the edge set of G (S) consists of all of the edges in E that have both endpoints in S. A subgraph 9 (S) is said to be a strongly connected component of G if we have (i) G (S) is strongly connected, i.e., there is a path in each direction between any two nodes and (ii) G (S) is maximal in the sense that no additional edges or nodes from can be included in the subgraph without breaking the property of being strongly connected. The collection of strongly connected components forms a partition of the nodes of G. A simple path is a path in a graph which does not have repeating nodes.
3.2 Collaboration Principles
The FL-PTs in the scenario under study have two features: (i) self-interest and (ii) competition among FL-PTs. Accordingly, there are two collaboration principles introduced below.
3.2.1 A Absence of free riders
While establishing collaboration relationships among FL-PTs, the following principle is used to guarantee the non non-existence of free rider.
Principle 1. For any FL-PT U, V, there exists a FL-PT uy E V that benefits es if and only if there exists at least one FL-PT U/ that can benefit from U,.
In this paper, all F1-PTs some FL-PTs. Let æ denote a partition of FL-PTs V into K mutually disjoint groups where π = {2,, .Sx) } satisfies: (i) S, C (ii) k = V and (ii) S; ns, = 0, Vi,j E 21, K). For any Sz E ग, the collaboration relationships among the FL-PTs of S₁ are established according to the subgraph (Sx) i.e., for any vi Uy E Sk, uy is contributor of v, if and only if there exists an edge (0;, 21) in the graph (Sx). Each FL-PTT U; E Sy is only concerned with the contributions of other FL-PTTs within the same St.
assumed to be self-interested and there exists competition among
Definition 1 (Coalitions). A partition स = (S1, ... SK} is said to be a set of coalitions if we have for any Sy. E with [S& 2 and U; E Sy that
En (v.) Way > 0 and Σ us (v:) W3ss > 0
(1)
In coalition St with |S&] > 2, each FL-PT v Se can both benefit and benefit from other members in St {v:)- Thus, in a set æ of coalitions, Principle ] will be realized since Eq. (1) is satisfied. If
|S&| = 1, the single FI- of Sn neither benefit nor benefit from any other FI.-PT.
3.2.2 Avoiding conflict of interest
To avoid conflict of interest, a FI.-PT will not contribute to its competitors (i.e., its enemies) and any FL-PTs that help its competitors (i.e., the friends of its enemies). Also, it doesn't 't hope to see

---

others help the supporters/friends of its competitors (in)directly, For any FL-PT Us let A, denote the supporter alliance of v defined as: A, = {rr V Ug reachable to v; in the data usage graph G. Let us consider any two competing FI.-PT's 2 and uy where (ry, 02) E Ee. To avoid conflict of interest, My will not contribute to any member of A.. Like [42], this is defined as the following principle by which it is strictly guaranteed that no FL-PTss will contribute to its competitors (in)directly
Principle 2. For any two competing FL-PTs vy and Vi Vi is unreachable to og in the data usage graph 9u-
3.3 Problem Description
Now, we propose proper problem formulation such that the resulting solution can well satisfy the FL-PTss needs and help them achieve the best possible ML model performances. All FL-PTs are divided into a set π of coalitions. For the FL-PTs of of a coalition Sk E x, their collaboration established according to the subgraph (Sx)) The utility u(Sa) of a coalition Se is defined as the sum of the edge weights of ). Formally, let (A(Ss) denotes the edge set of (s(S)); then, u(Sx) Die (e,,,e))EssS.. Wj, given FL-PT Us Sk, its utility is defined as (Sx) = Σ (ees,r)) Wa, where i2 is fixed; here, the utility the utilities of all its members, i.e., u(Sa)) = Ev.es, (x())). For two coalitions S₁ and Sy with Sy S S. (Sx) ) is a subgraph of Go (S))); then the utility of us in a larger coalition Sy is no smaller than its utility a smaller coalition St, i.e., u, (S < uy (5%).
relationships
coalition is the sum of
In this paper, our final problem is to find a partition л of FL.PPTs such that
Principles and are satisfied.
Subject to Principles [ no coalitions of π (i.e., no subset x' of T) can collaborate together and merged into larger coalition with higher utility; in other words, after merging these coalitions into larger one, no FL.-PT's in these coalitions x' increase their utilities under the larger coalition. Formally, let
II
CC
Σ u(Sk)=
U ELEN'
Sk
Principles 1 and 2 are satisfied by
U SRER!
Sk
StEn'
Then, II satisfies:
=== 0.
(2)
4 Solution
Now, we give an algorithm that satisfies Principles [ and 2 and Eq. (2). It is presented as Algorithm and its main idea is follows. Firstly, we find a partition = (Si, ... ,S of all FL-PTs such that the FL-PTs of each subset S, E â are independent of each other. Secondly, S, E ft is further partitioned into several subsets/coalitions, denoted as SCC, = 1 Sr.2. SSh } such that for all / E (1), Go (Sa.1) is strongly connected component of (S.)).. Thirdly, for any coalitions of - SCC,, we merge these coalitions into larger one if doing so achieves a higher coalition utility without violating Principles [ and
Below, detail Algorithm [ Firstly. we define 9.- (V, E,) the inverse of the competing graph Ge, i.e., any two nodes of G-- are adjacent and only if they are not adjacent in Ga: if there an edge between
nodes in 9 then these nodes are said to be independent of each other (line 2). In graph theory, a clique of an arbitrary undirected graph G = (V, F)) is a subset of nodes C in which any two nodes of S are connected by an edge in the subgraph G(S); maximal clique is a clique such that if it is extended by adding any other node, the resulting larger subgraph is not complete. We search for all maximal cliques within 9 which form a partition A of all FL-PTs V. Any two FL.-PTss in subset. St E are independent of each other. In line 3, the cliques of 9.- can be found by the classic -erboSer algorithm A
Secondly, for each S,, E â. we process the subgraph (S,) of the benefit graph. All strongly connected components of (S,) are founded by the classic Tarjan algorithm [43). The node sets of
5

---

Algorithm 1: Conflict-free Coalitions without Free Riders
Input:
The benefit graph Gi, the competing graph G. Output: The set ı of coalitions
// Record the set of coalitions found by this algorithm. Construct the inverse of Gc, denoted as ;
<- 0:
Find all maximal cliques of ge, denoted as म = = (i,,... Su by the Bron-Kerbosch
algorithm;
for h + 1 to do
Find strongly connected components of Gb (5,) by the Tarjan algorithm: // The node sets of the components of Go (Sx)
denoted SCC, S SS.yn
Let = {θ), 022 ...
,,, Jy - SCC, where Y - yh:
Construct by Definition 2 directed graph Zn and an undirected graph Z, whose node sets are // 6y is a node 2, and Z. but also represents subset of /* Below, the node in of 2n with |â] = is processed.
8 Let y 4- Y 1;
1/ y is the index of the new node e, to be constructed. TT, Zb, Ze, y) MergeCycle(T,, Zo. Zei y). presented Algorithm
10 T, Zb. Ze, y) +- MergePath(T, x 2s, Ze: y), presented as Algorithm
/ Belon, the edge (6), dv) of Z& with | 2 and [óp| 2 is proceased.
11 (T, Zo. Be.8) y) + MergeNeighbors(fr 2,, Ze, y). presented as Algorithm
SCC,
07
Pa.
0z
Benefit
Compete
π
0
SCC₂
4
D5
66
(a) (SCCA)? where H = 2.
(b) n
Figure 2: Illustration of Algorithm
all these components constitute a partition of S₁. denoted as SCC, (lines 4-5). Thirdly, we also use By to denote a set Sp.yn. .. C V and let π Uh= SCC,, {b, 02 ,,,,) where Y = - Uh. S and 1 By = V (line 6); then construct by Definition 2 two new graphs B, and Z. whose node sets are π (line 7).
Definition In the graph Zo, there a directed edge from to dy if and only if there exist two nodes Uj E ir and uj E by such that (vi, vz) is a directed edge in the benefit graph Gb- In the graph Ze. there an undirected edge between it and uy and only if there exist two nodes vs E it and Uy ûy such that (vi,0) is an undirected edge in the compeling graph Go. For any two coalitions fr and in of n. in is said to benefit (resp. benefü from) by if there is directed edge (ft, dv (resp. there is an undirected edge (f1, iv)
(vv. in)) in the graph Zn; in and by are said to be competitive if
the graph Z, and independent of each other otherwise.
In line 9, Algorithm 2is called where there is while-loop. In Algorithm 2 we check whether the loop condition in line 1 is satisfied or not; this loop condition will be called the cycle condition subsequently. As illustrated by 02 and i in Figure of this cycle. The following operations are taken to update the graph Z, and Z,: (i) the nodes X merged into new node ú, where all the edges in the graph Z, that point to (resp. point from) the nodes of X' change to point to (resp. point from) Cg. and (ii) the nodes X' 2, also exist in the graph Za; such nodes in Z, still merged into whose endpoints are the nodes X' change to become the edges whose endpoints are 0y. The above operations will be used
if "yes". let X {6 10= denote the nodes
node 0g. and all the edges in the graph Z,
routine and are given in Algorithm 3
In line 10, Algorithm 4 called where there are two while-loops. In Algorithm 4 we check whether the loop condition in the outer while loop is satisfied (line 1); this loop condition will be called the

---

Algorithm 2: MergeCycle(T, 26, Ze, v)
while there is node Sz. of 2, with [6y, I 1 such that (i) there is a cycle (0y1 Byzi"" Oy::
) in the graph Za that contains By, and (ii) the nodes By.. Uye of this cycle are independent of each other do // the cycle condition. (0y. 3b, Ze, y) <- Merge(X, n, Zb, Ze. v), presented as Algorithm 3 == (0 19=1
where
3 Return (T, 2b, Z);
Algorithm 3: Merge(XX, Zo, Ze: y)
8y Ue, Ex Bg, y y 1, f- X, and <- U (0y):
2 Add By into 2, as a new node, and all the edges in the graph Z- that point to (resp. point from) the nodes of X change point to (resp. point from) ey:
3 Add Ey into Z, as a new node, and all the edges in the graph Z, whose endpoints are the nodes of X change to become the edges whose endpoints are Ey:
Remove the nodes of X from both ZB and Zzi Return (0g, N, Zz, Ze: v);
Algorithm 4: MergePath(rr TT, 2o, Ze, y)
while there is node 0y. of 2s, with 16g such that (i) there a simple path
(iy1:".. :.y:"" -.y::.'. .By)) with uy 2 2 and 0y >> 2 and (ii) the nodes Og u of this simple path are independent of each other do 11 the path condition.
(By, Zb, Zeo y) t Merge(X, Zn. Zoo y) where X (60 1
3
(TT, Zb, Ze, y) &- MergeCycle(fr, 2,, Z,, y), presented as Algorithm
Return (T, Zn, y):
Algorithm 5: MergeNeighbors (T, Zv. Ze, y)
while there is edge (6t, 6v.) of Zo with |ôt] > 2 and [ such that in and by are independent of each other do // the node condition.
(Cy. RT Zp, Ze. y) t- Merge( (X, n, Zr, Zo, 1) where X' = {0r 6y J:
(T, Zb, Ze: y) MergeCycle(T. ,, Zo, Ze: y). presented Algorithm 2
(IT, Zb, Zo. y) + MergePath( (T, Zo, Ze, y). presented as Algorithm
5 Return (Zp, Ze,
path condition. If "yes", let X' ( denote the nodes of the simple path. Then, Algorithm 3 is called with the current X' as its input. Afterwards, Algorithm 2 is called here to remove that type of cycles defined in line of Algorithm 2 In line 11, Algorithm is called where there three while-loops. We check whether the loop condition in the outer while loop is satisfied (line I); this loop condition is called the node condition. As illustrated by ig and é, in Figure ([a), if "yes". let X' = { U { then, Algorithm 3is called with X' as its input. Afterwards, Algorithms 2 and 4 are called here guarantee that those types of cycles and simple paths in the cycle and path conditions don't 1 exist in the graph Z
The time complexity algorithm depends on the Bron-Kerbosch algorithm in line 3 and is (33//3)
[4]; please see Appendix A.. for detailed analysis. In line 1 of Algorithm@ or any two coalitions 8g. and dg.. are independent (of each other); in line 1 of Algorithm 5 the two coalitions i and öy are independent. Any two members from such two coalitions respectively will be independent by Definition 2 each coalition, all its members are independent. Thus, all members of the merged i are independent of each other. For any coalition i.. the collaboration relationships among its members are established according to the subgraph Go thus, we have two conclusions. Firstly, Principle 2 is realized since for the data usage graph G.. paths may exist only between two nodes
7

---

in coalition. Secondly, after the merge operation in line 2 of Algorithm 2 or for any 0y with By 1, can benefit from its direct predecessor and benefit its direct successor in the cycle or the simple path, satisfying Eq. TT. As result, we have the following proposition whose detailed proof can be found in Appendix A.2
Proposition Upon completion of Algorithm Principles
7 and 2 are realized.
Upon completion of Algorithm 1 the cycle condition, the path condition, and the node condition are not satisfied. Then, we have the following two conclusions: (i) when the cycle condition isn't t satisfied, we have for any coalition 8y. with 16,p1 without violating Principles I and ii and only if the path condition is satisfied in the graphs Z. and Zoi and (ii) When both the cycle condition and the path condition aren't I satisfied, we have for any coalition êy with |oy] > 2 that it can be merged with other coalitions without violating Principles I and 2 if and only if the node condition satisfied in the graphs Z, and Ze. The proofs of the With these two conclusions, we further have the
that it can be merged with other coalitions
above two conclusions are given in Appendix A.3
following proposition.
Proposition 2. Upon completion of Algorithmm
Eq.
holds.
5 Evaluation
5.1 Experimental Setup
Datasets & data heterogeneity. We conduct experiments on the CIFAR-I0 and CIFAR-100 datasets with different data heterogeneity settings (23. Also. we use weal-world dataset eICU (36) illustrate the practicality of FedEgoists. Both the CIFAR-10 and CIFAR-100 datasets contain 60,000 color images for image classification tasks but have different levels of complexity. CIFAR-10 images have 10 classes with 6,000 images per class, while CIFAR-100 is more complex and has 100 classes with only 600 images per class. We simulate the data heterogeneity by two typical approaches: (i) pathological distribution S 32 61]. where each FL-PT is randomly allocated 2 classes of images for CIFAR- 10 and 20 classes of images for CIFAR- 100, and (ii) Dirichlet distribution
41 57 62].
where a distribution vector de E I is drawn from the distribution Dir, (B) for each class and FL-PT 2; allocated a ye, proportion of data samples of class e; smaller 3 value results in higher data heterogeneity, and we set 3 0.5. Above, we use standard datasets and randomly divide each dataset into different parts that are used as the local data of different FL-PTs. We also use a real- world dataset that contains data directly from multiple different FL-PTs. Specifically, eICU is dataset collecting EHRs from many hospitals across the United States admitted to the intensive care unit (ICU). The task is predict mortality during hospitalization. More details on the way of processing these datasets are given in Appendix B.I
Comparison baselines. FedAvg 32 is widely recognized as vanilla FL algorithm. Multiple representative personalized FL (PFL) methods are used as baselines for comparison [11. FedProx
25 and SCAFFOLD [2] represent two typical approaches that make the aggregated model the CS close to the global optima and are two benchmarks in [24). In other baseline methods, different personalized models are trained for individual FL-PTs. pFedHN (38) and Me
440 represent two
approaches based on hypernetworks and meta-learning respectively. FedDisco (58 and Gradhraph
[57] are two approaches based on data complementarity. which are proposed recently. FedOra 48 is method that assesses whether FL-PT's generalization performance can benefit from knowledge transferred from others and maximizes it. The last baseline Local where each FL-PT simply takes local ML training without collaboration. These traditional FL approaches cannot address the issues of both self-interest and competition. Thus, we use the operations lines 1-4 of Algorithm Ioo generate a set of coalitions, denoted as _ SCC,, where SCC, = 1. ... Sp,y. J. If [S > 2, Sr represents group of independent FL-PTs that can collaborate together following Principles Tand 2 the above nine FL approaches are applied to each group of such FL-PT's to generate the baseline
results.
the hypernetwork technique 33 is used to compute the benefit graph G. and hypernetwork is constructed by multilayer perceptron (MLP). The specific way of generating Go is introduced in Appendix B.2 The used network structures are introduced in Appendix BB. There are 10 FL-PTs in the CIFAR-10, CIFAR-100 and eICU experiments.
Finally, like 5 26 42
8

---

Table 1: Accuracy comparisons (MTA) under different a on CIFARIO.
LOCAL
FEDAVG
FEDPROX
SCAFFOLD
PFEDME
PFEDHN
FEDDISCO PFEDGRAPH FEDORA
FEDEGOISTS
0.05
PAT 80.47.1..06
36.86.13..0 011.1.1..0666622......0 6.17 36.61 48.56.16.18 66.53 2.00 16.11
52.04.18.66 69.73. 81.35.0.000
0.05
Dir 61.59:0.553 14 46,94=2.12 46.76 44 44.6442.11 5..61 46.74
46.36+22.55 55.28=0.75 63.06:10.644
0.1
PAT 80.47::2.06
49 40:5 50 48. 19 48, 16
56
66 66.61
= 1.62 48. 1915,17
55.3544511
68.65-2.22
10.73::1.335
0.1 Dir 61.59+0..3 46.77=1
4.97 48.6142.02 46.65-22.44
5..1
18.56+11.9 .9
49.10+3.19
55.97=0.22
62.74::1.09
0.2 80.47.1..006
63.67 57.26 57.2441...44 79.27:11.35 76.08
57.
60.27:12.33
1.9
1.43
0.2 61.59:20.53 15
61.31
14.08+1443
53,85+1077
55.67=0.96
66.62:::1.23
0.3 PAT 80.47:42.006 $7.95:22.37
65.11
59.82:14.88
62.12:4-511
71.51=2.40
1.437
0.3 Dir 61.59:10.533 50,48:10.87
50.66.11.59
55.911.01
63.3910.89
0.4 PAT 80.47:22.06 58.4745.877
03.2744446 66866 611.0:1000000010.111111000300.3.
63.28
63.30-44.61
72.89=1667
82,54=0.30
0.4
Dir 61.59:1:0.33 50.14422.2 51.20=2.166 51.2331...11..22...0
51 0.94 51.
51.14:22.1 16 57.26-0.32
62.81110888
Table 2: Accuracy comparisons(MTA) under different a on CIFAR100.
LOCAL
FEDAVG
FEDPROX
SCAFFOLD
PFEDME
PFEDHN
FEDDISCO PFEDGRAPH FEDORA
PEDEGOISTS
0.05
16.2441..38
34.52.18.65 35.42 15.33:55 69.4 69.73 18 18. 0.9 69.76:16.56
15.4 35.78
29.98 1.07 35.4243.558 16.60:11.15 41.91=0.49 47.00:1
0.05
30.31::0.48
19.76:16.00 27.06=0.26 27.59::1.52
0.1
PAT 16.24
44 30, 30.17 45.56:00:45
42.78:00:6
42.63=1.04
1.5
0.1
Dir 30.31.10.48 20.25 20:251493
18.80
20.51110.98 17.45
18.87
18.88.14995
27.50_00.1
1.60
0.2 PXT 46.24:11.38 29.68:-4.12 28 22000052444.62555544... 28.55+44.34
188 28.38=0.71 29.05=-4.11
30.51+-4.03
41.63=1.65
50.21+22.24
0.2 Dir 30.31:10.48 0922411113300000000003 20.00:115..0000000.00000
19.93:10.38
20.17:10.35
27.24=0.36
1..86.11.533
0.3 PAT 16.24 40.24: 42,4::205554422222222222 42.57 44.34
42.44:4::
42.4840.48
41.72=1.98
16.38
0.3 Dir 30.31-10.48 25.56.10.32 22.50110002227311107777772222222 25.20.22225...222.
17.11
26.18:1169
27.43-0.20
34.30.10.4
0.4 PAT 46.24:1138 40.52:00.27 4..63=1003 41.71
449 30.18
4..73
11.66::110
42,94=0.25
48.16::1.61
0.4
Dir 30.31+00488 24.73:40.97 27.37=0.40 27.31
1.8 17.08=0.35 27.37:0040
27.1740.42 27.24-0.23
34.15-10.66
5.2 Benchmark Experiments: CIFAR-10 & CIFAR-100
We conduct experiments on CIFAR-10 and CIFAR-100 with competing graphs that are generated randomly. To simulate competition, the probability of two FL-PPTs competing against each other is set to a, thus generating random competing graph G. 42), which constrains the collaboration between some FL-PTTs; here, the probability that two FL-PPTs are independent of each other is 1-a. The value of determines the intensity of competition among FL-PPTs and a larger value reflects higher level of competing intensity among FL-PT'.
Firstly, we verify the effect of the competition intensity on the effectiveness of the proposed solution where we vary the value of a that takes different values in {0.05, 0.1, 0.2, 0.3, 0.4] and conducted the corresponding experiments. Given specific value of a, we conducted five trials to show the average performance. For the 1-th trial, a particular competing graph Ge, is randomly generated with the given a; then, the experiments for the baseline and proposed approaches are run; the performance of the proposed approach is denoted as Ta.l.p while the performance of the I-th baseline approach is denoted ra.1,i. i E f1, 9}. Given the value of a, we show the average performance of the five trials (i.e., Σ ta and Σi= roda i E 21, .......)). -9). The model performance is measured by the mean test accuracy (MTA), and the experimental results are presented in Tables [2 in the form of ttd: here, PAT and Dir represent the pathological and Dirichlet distributions respectively. It is observed from Tables 2 that on average. FedEgoists achieves a significant performance improvement when compared with all the baseline approaches. In Appendix [B.41]] we also illustrate the collaboration relationships among the ten FL-PTs. It is shown that FedEgoists can facilitate the collaboration among FL-PTss and thus achieves a better performance.
5
Secondly. we define a metric to evaluate the worst-case performance across the five trials. In each trial, the nine haseline approaches perform differently. We find an integer I" such that under the 1- trial, the baseline approaches achieve the best performance, i.e. 1* = arg max]e[1.5] (max To...) where max.e .9 fa,i, is the best performance of all the nine baseline approaches in the 1-th trial. In the trial that is the most advantageous to the baseline approaches, ma.,g((19) Pa. Ta,"" the performance improvement (or difference) of the best baseline approach to the proposed approach, which may be negative the proposed approach achieves better performance. Upon identifying I', we then detail the performance outcomes of both the proposed and baseline models during the 1" trial. For various values of a, ranging from 0.05 to 0.4, we calculate the performance difference, max Ta Ta, pp- The results for these computations, under each specified a.

---

Table 3: The worst-case performance of the proposed approach compared with the baseline ap- proaches.
0.05
Dir
PAT
CIFARIO
1000
0.002903
0.022900
0.025800 -0,000008
0.000624
0.0006030
0.028800 0.000020
0.005725
0.002219 -00000699
0.000100
CIFARIO0
0.000999
0.076002
0.011400
-0.000636
0.0009356
-0.032153
-0.027078
Table 4: eICU
AUC
LOCAL
FEDAYG
FEDPROX
SCAFFOLD
DFEDME
pEEDNN
REDDISCO
DEEDGRAPH
FEDORA
FedEgoists
13.64422.12
63.52422.40
80.4219.85
80.2429.92
$2.30219.79
4199419.14
60/48113007
80,4219.85
90.36:2.26
66.36219.28
01
67.9446.88
62.55416.49
57.03=16.62
57.21=16.68
46.00:34.96
76.61=14.77
63.76+14.97
59.6247.49
81.52=16.91
81.58:6.65
37.33+17.74
76.48+12.70
60.1376.77
60,38+6.64
36.48+27.59
79.62+16111 18 92.70+4.60
57,32+8.17
47.56+9.62
66.04+33.21
88888+111 16
67 (7.04426.74
78.74=15.66
18.87115:44
45.79432.04
56.35+26.55
80.38+18.24
78.09#7.48 48
15:1227.85
84,40=5.76
52.48+11 61
13.46+15.58
73.63+9.74
75.75=11.07
57,07+23.12
48.75422.68
10.1549.96
49.6145.31
48.9546.80
15.84=11.26
45
39,45+9.00 06
$7 00997966
61.94+9.13
61,70=9.12 12
55, 15:14922
52.55425. 12 53.0349.73
89.37+7.71
77.72+8.24
68.41=5.60
68.00132.66
77.6715.87
79.62:7.62
78.74=7.81
57,23:32.51
42.01:16.65
82.2616.41
08.80:0.76
98.55=1.18
56.8627.52
73.3647.08
71. 80.9.52
73.55=10.48
73.59=10.17
$6.6012.556
51.21a5.01
68.45=10.98
76.82:11.00
75.53a5.94
77.97=14.94
36.24+22.55
73.5542.70
77.47=1.80
77.43+3.66
61.22+10.49
46.71+16.08
65.0543.41
69.16+3.12
12.06=12.01
10.57
10.60
63 14+9.42
63,82+9.32
42.97+12.63
45,42+17:42
63 03.24+10:63
60.76+10.1 12
58.55=7.62
79.88-8.29
Avg
58.01
68.62
70.06
70.77
31.08
54.02
69.95
72.06
72.61
74.79
systematically presented in Table which shows that in the worst case, the proposed FedEgoists has a performance very close to the best performance of all the baseline approaches.
Finally, we also conduct experiments to show the effect of data heterogeneity and the related results can be found in Appendix BB4.2
5.3 Real-world Collaboration Example: eICU
there are ten hospitals in total, with {us) as large hospitals and {u.)i-m as small hospitals. Due to the extreme imbalance of data labels, where over 90% are negative labels, we use the AUC scores to evaluate the performance of the trained model. Suppose there are more than one large hospital located in the same city while small hospitals are dispersed in different rural areas with lower population densities; competition mainly occurs among large hospitals. We posit that competitive relationships exist between the following pairs of FL-PTs: (V1, Un), (V2, Vs). and (V2, v1). Table 4 presents our experimental results. In Appendix B.5 also illustrate the collaboration relationships among the 10 FL.sPTs. In the naive baseline methods, the formed coalitions 0g In the proposed method, larger coalitions formed so that their members can benefit from more FL-PPT's in the same coalition; the formed coalitions are {vo, U3, u...... ] and {vl, v2). Thus, FedEgoista can facilitate the collaboration among FL-PTs. In Table it is observed that ShatFeds FedEgoists achieves a better performance on the whole.
Following the setting in 5 2
(vo, vy va). {v1, 12) and (Vs, P';-
In addition to CIFAR-10, CIFAR-100 and eICU, we also conduct experiments the synthetic data like 42 and the related settings and results are presented in Appendix B.6
6 Conclusions
In this paper, we consider cross-silos FL where organizations in the business sector that engage in business activities are key sources of FL-PPTs. The resulting FL. ecosystem has two features: (i) self-interest and (ii) competition among FL-PTs. Correspondingly, two principles are proposed to build a sustainable FL ecosystem: (1) a FL-PT can benefit from the FL ecosystem if and only if it can benefit the FL ecosystem, and (2) FL-PT will not contribute its competitors as well as the supporters of its competitors. The Fl ecosystem needs to be tailored to realize these principles. In this paper, we propose an efficient solution that can well realize the two principles above. In the meantime, for any two coalitions without competition that can collaborate together, one coalition cannot increase its utility by collaborating with the other coalition. Extensive experiments over real-world datasets have demonstrated the effectiveness of the proposed solution compared to nine baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL.-PTss that engage in business activities.
10

---

Acknowledgments and Disclosure of Funding
This work is supported in part by the National Key R&D Program of China under Grant 2024YFE0200500. This research is supported in part by the National Natural Science Foundation of China under 62327801. This research is supported, in part, by the National Research Foundation Singapore and DSO National Laboratories under the AI Singapore Programme (No: AISG2-RP-2000 019); and the RIE 2020 Advanced Manufacturing and Engineering (AME) Programmatic Fund (No. A20) Singapore. This research/project is supported by the National Research Foundation, Singapore and Infocomm Media Development Authority under its Trust Tech Funding Initiative. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views National Research Foundation, Singapore and Infocomm Media Development Authority.
References
[1] Haris Aziz, Rahul Savani, and Hervé Moulin. Hedonic games. In Felix Brandt, Vincent Conitzer, Ulle Endriss, Jérôme Lang, and Ariel D. Procaccia, editors, Handbook of Computational Social Choice, page 356-376. Cambridge University Press, 2016. doi: 10.1017/ CB09781107446984001
[2] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint 1600.v/607.0e 2016.
[3] Ricarda B Bouncken, Johanna Gast, Sascha Kraus, and Marcel Bogers. Coopetition: a system- atic review, synthesis, and future research directions. Review of managerial science, 9:577-601, 2015.
[4] Bhaskar Ray Chaudhury, Linyi Li, Mintong Kang, Bo Li, and Ruta Mehta. Fairness in federated learning via core-stability. In Advances in Neural Information Processing Systems (NeurIPS'22) volume 35, pages 5738-5750, 2022.
[5] Sen Cui, Jian Liang, Weishen Pan, Kun Chen, Changshui Zhang, and Fei Wang. Collaboration equilibrium in federated learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'22), page 241-251, 2022.
[6] Qianlong Dang, Weifeng Gao, and Maoguo Gong. Dual transfer learning with generative filtering model for multiobjective multitasking optimization. Memetic Computing, 15(1):3-29, 2023.
[7] Shu Ding and Wei Wang. Collaborative learning by detecting collaboration partners. In Advances in Neural Information Processing Systems (NeurlPS 22), volume 35, pages 11629-15641,, 2022.
[8] NetworkX Documentation. all siappl_ppatsss....__ httts:
stable/rreferenc//alggrr httpa//aaaaeeeeaeeooooo///eeeeee awg.retts.ooottitttooooomm
all_ all__smmgll
html Accessed: 20-2-10-24.
[9] Kate Donahue and Jon Kleinberg. Model-sharing games: Analyzing federated learning under voluntary participation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(6):
5303-5311, May 2021.
[10] Randy Goebel, Han Yu, Boi Faltings, Lixin Fan, and Zehui Xiong. Trustworthy Federated Learning. Springer, Cham, 2023.
[11] Ronen Gradwohl and Moshe Tennenholtz. Coopetition against an amazon. Journal of Artificial Intelligence Research, 76: 16:1007-1116, 2023.
[12] Shangwei Guo, Tianwei Zhang, Guowen Xu, Han Yu, Tao Xiang, and Yang Liu. Byzantine- resilient decentralized stochastic gradient descent. IEEE Transactions on Circuits and Systems for Video Technology (TCVT), 32(6))4096-4106 2021.
[13] Xianjie Guo, Liping Yi, Xiaohu Wu, Kui Yu, and Gang Wang. Enhancing causal discovery in federated settings with limited local samples. International Workshop on Federated Foundation Models in Conjunction with NeurIPS 2024, 2024.
11

---

[14] Tiantian He, Yew Soon Ong. and Lu Bai. Learning conjoint attentions for graph neural nets.
Advances Neural Information Processing Systems, 34:2641 1-261,, 2021.
[15] Tiantian He, Yang Liu, Yew-Soon Ong. Xiaohu Wu, and Xin Luo. Polarized message-passing in graph neural networks. Artificial Intelligence, 331:104129, 2024.
[16] Chao Huang, Shuqi Ke, and Xin Liu. Duopoly business competition in cross-silo federated learning. IEEE Transactions on Network Science and Engineering, 11(1):340-351, 2024.
[17] Chao Huang, Ming Tang. Qian Ma, Jianwei Huang, and Xin Liu. Promoting collaboration in cross-silo federated learning: Challenges and opportunities. IEEE Communications Magazine, 62(4):82-88, 2024.
[18] Hyconji Hwang, Seongjun Yang, Dacyoung Kim, Radhika Dua, Jong- Yeup Kim, Eunho Yang, and Edward Choi. Towards the practical utility of federated learning in the medical domain, In Conference Health, Inference, and Learning, pages 163-181. PMLR, 2023.
[19] Donald B Johnson. Finding all the elementary circuits of directed graph. SIAM Journal on Computing, 4(1):77-84, 1975.
[20] Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Har- chaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi. Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konecný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Hang Qi, Daniel Ramage, Ramesh Raskar, Mariana Raykova, Dawn Song. Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Pranceth Vepakomma, Jianyu Wang. Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, and Sen Zhao. Advances and open problems in federated learning. Found. Trends Mach. Learn., 14(1-2):1-210, jun 2021. ISSN 1935-8237. doi: 10. 00.1561/2000000883
[21] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. Scaffold: Stochastic controlled averaging for federated learning. In International conference machine learning, pages 5132-5143. PMLR, 2020.
[22] Sai Praneeth Karimireddy, Wenshuo Guo, and Michael Jordan. Mechanisms that incentivize data sharing in federated learning. In the Workshop on Federated Learning: Recent Advances and New Challenges, in Conjunction with NeurlPS, 2022.
[23] Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[24] Qinbin Li, Yiqun Diao, Quan Chen, and Bingsheng He. Federated learning on non-iid data silos: An experimental study. In Proceedings the IEEE 38th International Conference Data Engineering (ICDE'22) pages 965-978, 2022.
[25] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. Federated optimization in heterogeneous networks. Proceedings of Machine learning and systems, 2:429-455 450, 2020.
[26] Zhilong Li, Xiaohu Wu, Xiaoli Tang, Tiantian He, Yew-Soon Ong, Mengmeng Chen, Qiqi Liu, Qicheng Lao, Xiaoxiao Li, and Han Yu. Benchmarking data heterogeneity evaluation approaches for personalized federated learning. In International Workshop on Federated Foundation Models in Conjunction with NeurIPS 2024, pages 1-9, 2024.
[27] Ken Liu, Shengyuan Hu, Steven Z, Wu, and Virginia Smith. On privacy and personalization in cross-silo federated learning. Advances in Neural Information Processing Systems, 35: 5925-5940, 2022.
12

---

[28] Ryan Wen Liu, Yu Guo, Jiangtian Nie, Qin Hu. Zehui Xiong, Han Yu, and Mohsen Guizani. Intelligent edge-enabled efficient multi-source data fusion for autonomous surface vehicles in maritime internet of things. IEEE Transactions Green Communications and Networking (TGCN), 17) 2022.
[29] Guodong Long. Yue Tan, Jing Jiang, and Chengqi Zhang. Federated learning for open banking. In Federated Learning, pages 250.254. Springer, 2020.
[30] Hassen Louati, Ali Louati. Slim Bechikh, and Elham Kariri. Joint filter and channel pruning of convolutional neural networks bi-level optimization problem. Memetic Computing, 16(1): 71-90, 2024.
[31] Lingjuan Lyu, Jiangshan Yu, Karthik Nandakumar, Yitong Li, Xingjun Ma, Jiong Jin, Han Yu, and Kee Siong Ng. Towards fair and privacy-preservnng federated deep models. IEEE Transactions on Parallel and Distributed Systems, 31 2541, 2020.
[32] Brendan McMahan, Eider Moore. Daniel Ramage, Seth Hampson, and Blaise Agueray Arcas. communication-efficcee. learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS' 17), pages 1273-1283 1282, 2017.
[33] Aviv Navon, Aviv Shamsian, Ethan Fetaya, and Gal Chechik. Learning the pareto front with hypernetworks. In International Conference on Learning Representations (ICLR'21) 2021.
[34] Limiao Ning, Junfei Dong, Rong Xiao, Kay Chen Tan, and Huajin Tang. Event-driven spiking neural networks with spike-based learning. Memetic Computing. 15(2):205-217, 2023.
[35] Martijn Oldenhof, Gergely Ács, Balázs Pejó, Ansgar Schuffenhauer, Nicholas Holway, Noé Sturm, Ame Dieckmann, Oliver Fortmeier, Eric Boniface, Clément Mayer, et al. Industry-scale orchestrated federated learning for drug discovery. In Proceedings of the AAAI Conference Artificial Intelligence, volume 37, pages 155841 2023.
[36] Tom J Pollard, Alistair EW Johnson, Jesse D Raffa, Leo A Celi, Roger G Mark, and Omar Badawi. The eicu collaborative research database, a freely available multi-center database for critical care research. Scientific data, 5(1):1 2018.
[37] Felix Sattler, agnostic distributed multitask optimization under privacy constraints. IEEE Transactions Neural Networks and Learning Systems, 32(8):3710-377 1022002 2020.
Müller, and Wojciech Samek. Clustered federated learning: Model-
[38] Aviv Shamsian, Aviv Navon, Ethan Fetaya, and Gal Chechik. Personalized federated learning using hypernetworks. In International Conference on Machine Learning, pages 9489-9502. PMLR, 2021.
[39] Chenxi Sun, Chao Huang. Biying Shou, and Jianwei Huang. Federated learning in competitive charging market. In 2023 IEEE PES Innovative Smart Grid Technologies Europe (ISGT EUROPE), pages 1-5, 2023.
[40] Canh T Dinh, Nguyen Tran, and Josh Nguyen. Personalized federated learning with moreau envelopes. Advances Neural Information Processing Systems, 33:21394-2144 2020.
[41] Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems, pages 1-17, 2022. doi: 10.1 10.1109/TNNLS. SIN.LS..222.3160699
[42] Shanli Tan, Hao Cheng, Xiaohu Wu, Han Yu, Tiantian He, Yew Soon Ong, Chongjun Wang, and Xiaofeng Tao. Fedcompetitors: Harmonious collaboration in federated learning with competing participants. Proceedings of the AAAI Conference on Artificial Intelligence, 38(14): 15231-15239, Mar. 2024.
[43] Robert Tarjan. Depth-first search and linear graph algorithms. SIAM journal on computing, 166,1461160, 1972.
13

---

[44] Etsuji Tomita, Akira Tanaka, and Haruhisa Takahashi. The worst-case time complexity for generating all maximal cliques and computational experiments. Theoretical computer science, 363(1):28 42, 2006.
[45] Nikita Tsoy and Nikola Konstantinov. Strategic data sharing between competitors. Advances in Neural Information Processing Systems, 36, 2024.
[46] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[47] Yansheng Wang, Yongxin Tong, Zimu Zhou, Ziyao Ren, Yi Xu, Guobin Wu, and Weifeng Lx. Fed-Itd: Towards cross-platform ride hailing via federated learning to dispatch. Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining (KDD'22) 22), pages 4079-4089, 2022.
[48] Jun Wu, Wenxuan Bao, Elizabeth Ainsworth, and Jingrui He. Personalized federated Iearn- ing with parameter propagation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2605, 2023.
[49] Xiaohu Wu and Patrick Loiseau. Algorithms for scheduling geadline-sensitiie malleable tasks. In Operations Research Forum, volume 5, page 30. Springer, 2024.
[50] Xiaohu Wu and Han Yu. Mars-fl: Enabling competitors to collaborate in federated learning. IEEE Transactions on Big Data, pages 1-11, 2022.
[51] Xiaohu Wu, Francesco De Pellegrini, Guanyu Gao, and Giuliano Casale. A framework for allocating server time spot and on-demand services in cloud computing. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), 4(4):1-31, 4(4)1113, 2019.
[52] Xiaohu WU, Francesco De Pellegrini, and Giuliano Casale. Delay and price differentiation in cloud computing: A service model, supporting architectures, and performance. ACM Transactions on Modeling and Performance Evaluation Computing Systems, 8(3):1-40, 2023.
[53] Liu Yang, Ben Tan, Vincent W. Zheng, Kai Chen, and Qiang Yang. Federated recommendation systems. In Federated Learning: Privacy and Incentive, pages 225-239. Springer, 2020.
[54] Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong. Federated machine learning: concept and applications. ACM Transactions on Intelligent Systems and Technology, 10(2):12:1-12:19, 2019.
[55] Qiang Yang, Lixin Fan, and Han Yu. Federated Learning: Privacy and Incentive. Springer, Cham, 2020.
[56] Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. Federated Learning. Springer, Cham, 2020.
[57] Rui Ye, Zhenyang Ni, Fangzhao Wu, Siheng Chen, and Yanfeng Wang. Personalized federated learning with inferred collaboration graphs. In International Conference on Machine Learning. pages 39801-39817 PMLR, 2023.
[58] Rui Ye, Mingkai Xu, Jianyu Wang, Chenxin Xu, Siheng Chen, and Yanfeng Wang, Feddisco: Federated learning with discrepancy- aware collaboration. In International Conference on Machine Learning, pages 39879-39900 PMLR, 2023.
[59] Han Yu, Chunyan Miao, Bo An, Zhiqi Shen, and Cyril Leung. Reputation- aware task allocation for human trustees. In The 13th International Conference Autonomous Agents and Multi- Agent Systems (AAMAS' 14), pages 357-364, 2014.
[60] Han Yu, Chunyan Miao, Cyril Leung, Yiqiang Chen, Simon Fauvel, Victor R. Lesser, and Qiang Yang. Mitigating herding in hierarchical crowdsourcing networks. Scientific Reports, 6(4), 2016.
14

---

[61] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan. Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 1 11237-11244 2023.
[62] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heteroge- neous federated learning. In International conference on machine learning, pages 1 12889, PMI.R 2021.
A Theoretical Analysis
A.1 The Time Complexity of Algorithm
Now, we analyze the time complexity of Algorithm 1 Firstly, the construction of g. in line 2 involves two nested for loops, resulting in complexity of (n²). In line 3, the Bron- Kerbosch algorithm is used, which has time complexity of (33/33) [44] In lines 4-5, there is for-loop; each iteration, the Tarjan's is algorithm is executed on (S,) with a time complexity of Omm, + ex) 43, where my the number of nodes in (S,) and e₁ denotes the number of its edges; thus, the aggregate time complexity is On + [.. The time complexity of forming a union of sets i in line 6 is O(Y). The construction of 2, and Z, in line involves two nested loops to check the existence of edges between different node sets; at each iteration, the time complexity is ((6g)16y))) where y. V E [1, Y]; the total time complexity in line 7 is 2 where [0yg &
Initially, there are Y elements in the set π f8g, 02. .... By where Y & i is the node sets of 2g and 2g. In lines 9, 10 and 11 of Algorithm [] Algorithms 2 and 5 are sequentially called. In each of Algorithms 2 4 and 5 there is at least one while loop and the loop conditions are about the features of a cycle, a simple path or two neighboring nodes in the graph Z, At each iteration of these while loops, at least two nodes will be merged into new node 6y with multiple nodes of removed. So, there are a total of at most Y - 1 iterations in all these while loops and we always have [T| < m; afterwards, Algorithm ends.
In Algorithms 2 and 5 Algorithm 3 is called. The time complexity of line in Algorithm is O(xx)). In lines 2-4, 2n and Z, are reconstructed and the complexity is O(d where d is
the maximum of the in-degrees and out-degrees of all nodes of X in the graphs 2n and 2, and is upperly bounded by |r], which is no larger than n. To sum up, the total complexity of Algorithm is (ml)21)) < On²))
In Algorithm there is a while loop. In line I, all cycles can be found by the Johnson's 's algorithm [19] which is implemented in a Python library and has a time complexity of O ([|x] + F F2
O ((n + Ec 1)) where E E(Zs) denotes the number of edges in the graph Z, and e denotes an upper bound of the total number of cycles discovered. For each cycle, we check whether any two of its nodes are competitive, which has time complexity ((Tr|² < O(n²); in the meantime, we check whether there is a node 0g in this cycle with 10,.1 = 1, which has a time complexity ((()) O()). So, the time complexity of checking the cycle condition in line is O ((r² + E(Z)))(c + 1)). At each iteration of the while loop (line 2), Algorithm iss called and the time complexity of Algorithm 3 is (nz)). The number of iterations is upperly bounded by n. Thus, the complexity of Algorithm is no larger than
O ((n + (c 1)) + O(cn²)
O (n(n² E(Z,))(c+ 1)) O(fi (n, E E(Zo),c) c)) where
fi (m, E(Z,), c) = nan²2 + E((z))(c+1),
(3)
In Algorithm there is an outer while loop. In line 1, for any two nodes 6n1 and 6y.. we check whether they are such that 16.1l >> 2 and |0 >> 2, which has time complexity of (( <O(")).
For each pair of ig, and 6y.. with |0 | > 2 and |0y1 >> 2, we can use the modified DFS algorithm to find all the simple paths from 6y, to Gg [8. This algorithm is implemented in a Python library and has a time complexity of [([[T] E((,)))) & O((n+ E(Z.))T) where denotes an upper bound of the total number of simple paths between two nodes. For each path, we check whether any two of its nodes are competitive, which has time complexity ((T)?) < O(n²); in the meantime, we check whether there node by in this path with j = 1, which has a time complexity ((Tl) & On)). So, the time complexity of checking the path condition in line is
15

---

+ ((2))) - 0 ((22+ E(()))) = O E(z)), 7)) where
[2 ((z,) (n² B(Z)))
At each iteration of the while loop, the operations in lines 2-3 exccuted. In line 2, Algorithm called and the time complexity of Algorithm 3 is (n²)). In line 3, Algorithm 2 is called with time complexity O (fi (n, (,),, c)). The number of iterations is upperly bounded by n. Thus, the complexity of Algorithm 4 is O (n(fi(n, E(Z,), c) + f2(n, E())),)))
In Algorithm 5 the time complexity of checking the node condition in line has a time complexity of ((E(())))))) At each iteration, the operations lines 2-4 are executed. In lines 2-4, Algorithms and 4 are called sequentially. As analyzed above, these operations have time complexity no larger (,),cc c) + [2 ( r))) The number of iterations is upperly bounded by n.
than O (n((i (n, E 2)),),
Thus, the complexity of Algorithm 5 no larger than O (f E (ZZ)), c) + f2(n, E(Z,), r)))
To sum up, since complete graph of n nodes has total of n - 1) that is an upper bound of E(Z,).
where π is the node set of 2,, and |T| < n. Further, we have
(n, () f2 7) n
< (c+
(5)
(6)
The worst-case time complexity of the operations lines 9-11 depends on Algorithm and is no larger than
O (f,
+ f2(n, E(2,). r))) s s (n
Finally, if e and 7 can be upperly bounded by constants, then the time complexity of Algorithm ! depends on the Kron-Kerbosee algorithm in line and (33/33. We note that this paper studies cross-silo FL in business sectors where there are typically limited number of FL-PTss (c.g., 2 to 100)
16
27
A.2 Proof of Proposition 1
A graph G is said to be strongly connected if it contains a directed path from U; to új and directed path from My vn for every pair of nodes i; and by of G. Specially, graph that contains only single node said to be strongly connected trivially.
Lemma 1. Principle is realized when Eq. is satisfied.
Proof. Only the FL-PTss in the same coalition may collaborate. For any FL-PT E V, there exists coalition S₂ E π such that U; e Sx. When [Sp] = 1, the only FL-PT v; neither receives nor provides benefits any other FL-PT, thereby satisfying Principle trivially. When [S,| >> 2. U2 engages in reciprocal exchange of benefits with the other members of Sx - {wz), which is guaranteed by Eq. (1)
Thus, v. both contributes and benefits from other members and Principle [ satisfied.
Lemma 2. Upon completion of lines 1-6 of Algorithm 7 we have for any é E a with |& > 2 that
(i) all FL-PT's of the coalition on are independent of each other and (ii) Eq. 7 holds for any FL-PTT
0/ E 6z.
Proof. At line 3, we obtain all maximal cliques of the inverse of Ge, denoted as â {S. SH
The nodes of Sr are independent of each other. At line 5, all the strongly connected components of Go (Sh) are computed, and the node sets of the components of 9b Sh are denoted as SCC,, =
{ $ SS,,.,) } where S₁ = U Sr.1. Thus, the FL-PTs of each St,, are still independent of
each other. In line 6, (01, 02 )r) Ü SCCA where = DH Eb=1 Uh. Thus, the first point
h=1
of Lemma 2 holds. The subgraph (6x) is strongly connected. If | 2, we have for any node
22 E 6, that there are nodes of in that can be reachable to and from v;. Thus, the second point of Lemma holds.
In lines 7-11, some coalitions of (0r, 02, ... öy may be merged as a larger coalition by Algorithm
3 which occurs in lines 9, 10 and 11 of Algorithm I where Algorithms 4 and are called,
16

---

are called sequentially in Algorithm [5] The merge operation occurs (i.e., Algorithm 3 executed) if the cycle condition. the path condition and the node condition are satisfied in Algorithms 2 4 and 5 respectively. Upon
respectively; Algorithm2ij is also called in Algorithm 4 Algorithms
and and
each completion of Algorithm []] и updated: some coalitions will be merged into new coalition ig that is further added to æ, and then be removed from æ. Like Lemma we still have the following conclusion.
Lemma 3. At the beginning of each execution of Algorithm suppose we have for any 01 E π with [6/] > 2 that (i) all FL-PTs of the coalition in are independent of each other and (ii) Eq. 1 holds for
any FL-PT oe E 0j. Then, upon each completion of Algorithm 3 we still have for any in E a with | 3 2 that (i) all FL-PPTs of the coalition in are independent of each other and (ii) Eq. 1 holds for
any FI-PT U2 E 61.
Proof. Algorithm 3 is called if the cycle condition, the path condition and the node condition are satisfied in Algorithms
and respectivvl.
In line 1 of Algorithms 2 or any two coalitions by and f are independent of each other; by Definition 2 any two members from by and öyg respectively are independent of each other. In the meantime, Tor any 0y E ग,, any two members of Ey are independent of each other. Thus, upon completion of the merge operation in line 2 of Algorithms or 6yy "yg:"". ... Uyo are merged into a new coalition 8y. and all FL-PTs of By are independent of each other. This completes the proof of the first point of Lemma 3 when it comes to Algorithms 2 and 4
Similarly, in line of Algorithm 5 any two members from ún and éy respectively are independent of each other. In the meantime, for any 622 т,, any two members of fgg are independent of each other. Thus, upon completion of the merge operation in line of Algorithm|55 ú and by are merged into a new coalition ey. and all FL-PTss of fyy are independent of each other. This completes the proof of the first point of Lemma when it comes to Algorithm [
The collaboration relationships among the members of fy are established exactly according to the subgraph Go (.)) In line I of Algorithms or for any / E [1, 0). if |6 >> 2, the members of 6., already satisfy Eq. (I. Let us consider any two nodes 6 and By in the graph Z,. By Definition 2 if there an edge from G7 to Uy in the graph Zz. then there is node of 6. that benefits from node of
6 in the benefit graph Go. If [0 - 1, the single node in cy will benefit its direct successor in the cycle (resp. the simple path) and benefit from its direct predecessor in this cycle (resp. the simple
path); thus the node of fyy also satisfies Eq. (T). This completes the proof of the second point of Lemma
3 when it comes to Algorithms 2 and 4 In line of Algorithm since |èr > 2 and [0r >
the members of i and fy already satisfy Eq. This completes the proof of the second point of
Lemma 3 when it comes to Algorithm 5
Finally, by Lemma 2 we have the following conclusion holds upon completion of lines 1-6 of Algorithm|] [ for any in π with |6:] > 2, (i) all FL-PTs of the coalition úg are independent of each other and (ii) Eq. (T) holds for any FL-PT Uj E ōj. Afterwards, Algorithm may be executed multiple times by the call by Algorithms
2 4 and upon each completion of Algorithm B . is updated such that some of its coalitions are removed from π and merged into larger one that is added to T. By Lemma 33] upon completion of Algorithm[I [ the conclusion above still holds. Since Eq. ① holds, Principle is realized by Lemma I For any 6r E т,, the collaboration relationships among the members of coalition éy are established exactly according to the subgraph Go (fr). Each FL.-PT e of a coalition only collaborate with other FL.-PTss within the same coalition, The data usage graph 9,, in fact consists of multiple separated subgraphs (ūz), i.e. 9, = (Go(6) âg E m). In the graph G.., paths may exist only between two FL-PT's of the same coalition, and any two FL-PTs in the same coalition are independent of each other. Thus, Principle is realized.
A.3 Proof of Proposition 2
After finishing the operations in line of Algorithm [ the loop condition in line of Algorithm (i.e., the cycle condition) is not satisfied. The execution in line 10 of Algorithm starts. Before executing the first iteration of the while loop of Algorithm 4 if the loop condition (i.e., the path condition) is not satisfied, then AggorihhmFeend with both the cycle condition and the path condition violated. If Algorithm I ends after executing several iterations, then the cycle condition (line 3) and the path condition (line 1) are both violated.
17

---

Subject to Principles | and 2 no coalitions of x (i.e., no subset '' of α) can collaborate together and be merged into a larger coalition with higher utilityy; in other words, after merging these coalitions into a larger one, no FL,-PTs in these coalitions n' increase their utilities under the larger coalition. Formally, like what we define in Section 3.3 let
==
π
Σ Pz En'
u((ai)
U En:
8y
Principles and are
are satisfied by
U ie
Pay Ex'
Given a graph 2g where the cycle and path conditions are not satisfied, we have the following conclusion.
Lemma 4. Suppose the graph Zo is a graph without any such cycle that (i) the nodes of this cycle are all independent of each other and (ii) there exists a node Ūy in this cycle with |ox| = 1. Let us consider an arbitrary node Uy. of Z6 with 10 1; each node By. also represents a coalition; then, we have
The coalition By: can benefit or benefit from other coalitions by being merged with them without violating P Principles 2 aandippessnnn if there exists a path (Ty, nodes of this path are independent of each other.
there exists non-empty x' E TI such that By: E a') if and only Gys.... yy in the graph 2, with 0y 3 and By 2 2 and the
Proof. Firstly, we prove the "only if" direction by contradiction. For a coalition by. that contains only one FL-PT/node, it can collaborate with other coalitions without violating Principle only if there are two other coalitions y.. and 6y.., that can both benefit and benefit from it; by Definition 2 in the case that 02:-1 and Waa denote the same coalition, there is a cycle in the graph 2, between U Suppose there are some coalitions that can collaborate with
fy and i1:-11 where Gy fyg without violating Principles T and (i.e., there exists a non-empty n' E IT such that 6g2 E m'); correspondingly, in the graph Zo. there will be path of the form (ōg).... 0 Ug.s 0g 0g in which coalition will benefit its successor; the nodes of this path are also independent of each other according to the graph Z. in order be able collaborate together without violating Principle 2 Then, this path is simple and not a cycle (i.e., Sy. j 0gg)): otherwise, it is a cycle that has two nodes that compete against each other. In this path, all intermediate nodes can both benefit and benefit from other nodes. Each of 0g. and fgg also represents coalition and should contain at least two nodes of V: then, each node 0g. and 6z already satisfies Eq. ( thus maintaining Principle T Otherwise, if j |= 1, then the single node of igg cannot benefit from others; if Mgall = 1, then the single node of Uyo cannot benefit others. These two cases violate Principle This completes the proof of the "only if" direction.
Secondly, we prove the "if" direction. If there exists path (by)..... 0 Ugg) in the graph Z, with f0,1 I > 2 and |êyo > 2 and the nodes of this path are independent of each other, then the coalitions that these nodes the paths represent can be merged into a new subset úy of nodes of V; doing so doesn't 't violate Principle 2 i.e., the edge set of G. is empty. As shown above, each node in 6g and 0gu already satisfies Eq. I since any two nodes of GO())) or G0()) are connected. Also, in the graph Zz. any intermediate node can both benefit and benefit from other nodes in this path without violating Principle 1 Finally, let n' = {in ........ Uy ... Sy }. For any E [2, 0). there exists at least one FL-PPT e/ of U2g, that can benefit from some FI. in fg.. which is denoted as "y by Definition 2 Before merging the coalitions of n², the FL-PTT v; of a coalition ig, E n' only collaborates with the other FL-PT's with the same coalition 6n and the collaboration relationship is established according to Go Let 6, = UR.,, ex' Sg. After merging the coalitions of n' into a larger coalition ey. FL-PT v, can additionally benefit from FL-PT uy i.e., u4(0g)) ) < (,,ū)); in the subgraph Go (1,), ii can benefit from all FL-PTs of bg such that they point to v by directed edges.
This completes the proof of the "if" direction.
After finishing the operations in line 10 of Algorithm T. the cycle condition and the path condition cannot be satisfied. Then, the execution in line 11 of Algorithm [ starts. Before executing the first
iteration of the while loop of Algorithm 5 if the loop condition (i.e., the node condition) is not 5lens
satisfied, then Algorithm ends with the cycle, path and node conditions violated. If Algorithm 5 ends after executing several iterations, then the cycle and path conditions (lines 3-4) and the node condition (line 1) are violated.
18

---

Subject to Principles | and 2 no coalitions of x (i.e., no subset T' of α) can collaborate together and be merged into a larger coalition with higher utilityy; in other words, after merging these coalitions into a larger one, no FL,-PTs in these coalitions x' increase their utilities under the larger coalition. Formally, like what we define in Section 3.3 let
II
CTI
E f'
u(ór) <
U
6
Principles I and 2 are satisfied by U 0r
Given graph Zb where the cycle, path and node conditions are not satisfied, we have the following conclusion.
Lemma 5. Suppose the graph Zo is a graph without any such path that (i) the nodes of this path are all independent of each other and (ii) this path is either a cycle in which there exists node 6y
with [gy] 1, or simple path of the form (fgr: .... Ggg.
(gg)) with löy. = 1, 10 > 2 and
1:y0 > 2. Let us consider an arbitrary node in of Zo with (og 3 2: each node by also represents a coalition; then, we have
The coalition in can benefit or benefit from other coalitions by being merged with them without violating Principles 2 (i.e.. there exists a non-empty n' E 11 such that it E T') if and only if there exists another node by with [yy| > 2 that connects Or by an edge in the graph Za and the two nodes in and by are independent of each other.
Proof. Firstly, we prove the "if" direction. If there cxists another node by with |0p| > 2 that connects
it by edge in the graph 2, and the two nodes é and by are independent of each other, then the
coalitions that these two nodes in 2, represent can be merged into a new subset By of nodes of y:
doing so doesn't 't violate Principle b Definition 2 Each FL-PT in êt and êy already satisfies Eq. (T)
since any two FL-PTTs of (6, or Go are connected. Thus, any node in the merged set 6, still
satisfies Principle Finally we have n' {ūr, )r This completes the proof of the "if" direction.
Secondly, we prove the "only if" direction. For a coalition Byg that contains only one FL-PT/node, can collaborate with other coalitions without violating Principle[oonly if there are two other coalitions fg._, and oy... that can both benefit and benefit from it. Suppose there are some coalitions that collaborate with oy without violating Principles I Tand 2 (i.c. there exists non-empty x' I1 such that 6y E π''x correspondingly, in the graph 2o, there will be a path of the form (6g. 092s' ). (By.: Uy(+1: : ..... ). or (Ugn>'.. -1 By., 0g successor; the nodes of this path are also independent of each other according to the graph Z, in order be able to collaborate together without violating Principle 2 Then, by the assumptions in Lemma 5) this path is a simple path in which there doesn't node such that (i) the coalition that it represents contains only one node of V and (ii) there don't 't exist one of its upstream nodes and one of its downstream nodes whose corresponding coalitions both contain at least two nodes V. This means that the coalitions that the nodes of this path represent all contain at least nodes. This completes the proof of the "only if" direction.
(gg.) in which coalition will benefit its
As shown above. upon completion of Algorithm D the cycle, path and node conditions are not satisfied in the graph Zg. Firstly, by Lemma 4 for arbitrary multiple coalitions which there is at least one coalition that contains only one FL-PTT they cannot be merged into larger coalition without violating Principles [ []d Secondly, by Lemmaß] for arbitrary multiple coalitions each of which has least two FL-PTs, they cannot be merged into larger coalition without violating Principles I and 2 Thus, Proposition holds.
B
More Experimental Details
B.1 Data Processing
CIFAR-10 & CIFAR-100. In both CIFAR-IO and CIFAR-100, there are a total of 50,000 images for training and 10,000 images for testing. In CIFAR- 10, each class has 5000 training images and 1000 testing images. In 100, each class has 500 training images and 100 testing images. In both the CIFAR-10 10 and CIFAR- experiments, we split the data into 10 FL-PTs, and the training data for each FL-PT is divided into a training set (90%) and a validation set (10%). The label distribution of CIFAR-10 is illustrated in Figure 3 In Figure 3 the label distribution across different FL.-TT/FLL-PTT indicates that there is significant variance the total number of samples per FI.-PT, and the data is imbalanced. Similarly, the label distribution of CIFAR- 100 is illustrated in Figure
19

---

Pathologiezl distribution
Dirichiet distribtion
Chore
Cham
Cheee
Gler
4lerk4
Cli
Clian 5
Cliex
Cliat
Clian
Cleat 10
Cliar
Figure 3: Label distribution of CIFAR-IO. Colors indicate the labels for each FL.PPT/FL-PT.
Pathological distribution
Disichler distribution
Chers
Chest
Cheme
Clien
Cimm
Climi
Cliar
5
Cliss
Clicst
Clonk
Clies
9
Clisst
10
Clier
Figure 4: Label distribution of CIFAR-100. Colors indicate the labels for each FL-PT/FL-PPT.
eICU. We process the eICU dataset following the data pre-processing steps in 18, 30 generating JSON files that include hospital identifiers. We divided the training, validation, and testing sets in ratio of 7:1.5:1.5, with each hospital being treated as an independent FL-PT. We chose binary classification task using the eICU database, aiming predict patient mortality within 48 hours of ICU admission(mort 48h) based on the initial data provided.
finally
B.2 The Hypernetwork Technique for Generating the Benefit Graph Go
The local model information is mainly leveraged in the calculation of Go- Specifically, there FL-PTs. Each FL-PT i; has risk/loss function & R" - R,, Given a learned hypothesis E H. let the loss vector ((h) .e,, la) represent the utility loss of the FL-PTs under the hypothesis h. The hypothesis is considered Pareto solution if there is no other hypothesis & that dominates h, i.e.,
Åh' H.s.t.Vi. 4(h) & (h) and Bj e,())) < (h).
Let = (fI..... P.)) E TR. denote preference vector which denotes the weight of the objective local model loss that is normalized with En F and ra 2 0, Vk (I, The hypernetwork HN takes as input and outputs Pareto solution h, i.e.
HN (0, r),
where & denotes the parameters of the hypernetwork [23]. For each FL-PT vy linear scalarization be used. Like [5,30] optimal preference vector r (r),1 ... ".,.) is determined to generate the hypothesis hy that minimizes the loss with the data D, This is expressed as
h = (N, r;) where rf = argmin, (
(10)
20

---

VO
V1
8
V2
V
V
V6
(a) The Competing Graph Ge-
V2
VO
V
V
Vg
(b) The of coalitions in the baseline algorithms.
(c) The of coalitions by Algorithm
Figure 5: Illustration of Coalitions under CIFAR-100
For each FL-PT i, the value of ri,j is used as an estimate to the weight of uy to ez defines a directed weighted graph, i.e. the benefit graph Go.
5 26
{r;
B.3
Architecture Details
The network architectures used in different datasets are introduced below 5
4
15
30
B.3.1
CIFAR-10 & CIFAR-100
When comes specific dataset, all approaches have the same network structure for each FL-PT to execute the learning tasks.
Hypernetwork. We construct hypernetwork that employs a 2-layer hidden MLP to generate the parameters of the target network.
The target network passes through two convolutional layers, each followed by 2 x 2 max pooling layer. Afterward, data flattened and further processed through two fully connected layers, each followed by LeakyReLU activation function, finally yielding the prediction results.
CIFAR-10.
1000. Initially, the input data passes through two convolutional layers, activated by ReLU and each followed by 2x2 max pooling layer to reduce spatial dimensions and highlight important features. After these convolutional and pooling layers, the data is flattened to prepare for dense processing. Subsequently, it is fed into two fully connected layers, each incorporating ReLU activation function add non-linearity and improve learning capabilities. The output is finally produced by a linear layer.
B.3.2 eTCU
We employ single-layer MIP hypernetwork, while the target network utilizes a Transformer classifier [46] with Layer Normalization (2).
21

---

Table 5: Accuracy comparisons under different of Dirichlet distribution
CIFAR-10
CIFAR-100
0.01
0.1
0.5
0.01
0.1
0.5
0.80
FEDAVE
85.22 = 0.74
69.68
1: 3.24
52.25
+ 1.10
50.13
1 1.74
40.05
2.11
19.85
1.33
FEDPROX
0.97
70.17
50.35
35.25
20.71
0.37
SCAFFOLD
85.48 = 0.79
70.11 1.50
50.72
0.99
55.37 = 1.89
42.40
= 2.12
20.61
0.60
PFEDME
86.07 = 7.34
70.90 1.69
50.20 + 0.99
53.67 - 1.07
45.72 =
20.50
0.38
PFEDIN
85.57 1.65
0.56
47.60
33.73
23.72
0.91
PEDDISCO
85.52 0.82
70.21 + 1.60
50.64
1.65 1.65
$5.25
1.68
42.38
= 2.02
20.54
+ 0.50
PFEDGRAPH
0.97
1.29
2,00
FEDORA
$6.60 = 0.07
76.22 22 2.07
55.67 + 0.96
55.70
0.61
43.72 0.79
26.78
1 1.06
FEDEGOISTS
86.91 0.07
1.54 78.03
63,18
1,45
58.06
1.19
47.72
3,90
33.47
1.75
B.4
More Experiments on CIFAR-10 & CIFAR-100
B.4.1 Illustrating the Collaboration Relationships in the Benchmark Experiments
Now, give representative example illustrate the collaboration relationships among the 10 FL- PTs and present more details on the experimental results. The experiments are taken with CIFAR-100 100 under the Dirichlet distribution setting. The competing relationships exist in the following pairs of two nodes: (vo, Va va).(ro (v1 Un), ) (02; 17), (ry, (1). 07). (14,, 0g), (14. 0g), (#s, 07), and (vs, Us). This illustrated in Figure 5ja). The final collaboration relationships among FL-PPTs are illustrated in Figure (5) Figure b)b) illustrates the results for FedEgoists while Figure illustrates the results for the baseline methods. As shown, FedEgoists can facilitate the collaboration among FL-PTs and thus achieves a better performance.
B.4.2 The Effect of Data Heterogeneity
Data heterogeneity is typically simulated by pathological or Dirichlet distribution. For example, the pathological distribution is used and Dirichlet distributions respectively. In addition, We also verify the effect of the data heterogeneity level on the effectiveness of the proposed solution. Specifically, let denote the number of classes. In Dirichlet distribution, there is a distribution vector E R is drawn from the Dirichlet distribution (3) for each class e and FI.-PT v; allocated Ge,i proportion of data samples of class e; smaller 3 value results in higher data heterogeneity. We vary the value of / that takes different values in (0.01, 0.1, 0.5) and conducted the corresponding experiments. The related experimental results presented in Table 5] It is observed that, in some cases of high data heterogeneity, FedEgoista achieves a performance close to the Local approach; overall, FedEgoiste performs the best when is compared to the baseline approaches.
42 In this paper, we conducted experiments on pathological
B.5 Real-world Collaboration Example
Figure ((a) illustrates the generated benefit graph Go. Fig illustrates the final set и of coalitions returned by Algorithm [ while Figure (c)) illustrates the hnal set of coalitions of the baseline methods. Still, it is observed that, FedEgoists can facilitate the collaboration among FL-PTs and thus achieves a better performance.
B.6 Synthetic Data
We present our experimental results with synthetic data across different scenarios with fixed competing
graphs like 42 There are 8 FL-PTs. The synthetic data are generated by z ~ 4[-1.0, 1.0). The noise
|0000 0.52| is added to each label. Given the FL-PT vú the grand truth weights M4 = v + Ti.1 are sampled as 20.00, 1.0] and ri, ~ |[0.0.pp p²) where / = 1, 3; p² measures data distribution discrepancy among FL-PTs. We set p = 0.01, which means that the generated data are weakly non-iid in terms of sample features and labels. We construct the hypernetwork using 1-layer MI. for training the Pareto Front of all objectives. The target network consists of a two-layer linear network. After the first linear layer, Leaky ReLU activation function is applied.
22

---

2
3
4
VI
VO
1/
(a) The Benefit Graph Go-
V
3
V
V
2
4
V1
V
4
V1
V5
VO
V
VO
V6
Vg
V
V
V
V
(b) The set ı of coalitions in the baseline algorithms.
(c) The set of coalitions by Algorithm
Figure 6: Real-world Collaboration Example
Weakly Non-IID setting. The same regression task is learned by all FL-PTs and the synthetic labels are defined as:
y Σ kiml Lh +e.
(11)
FL-PTss U1, U2, U. and 0g have 2000 samples whose amount is large, while the other FL-PTs have 100 samples whose amount is small. Thus, there exists quantity skew, i.e., a significant difference in the sample quantities of FL-PT'. Two large FL-PTs ei and are independent and competes with the other two large FL-PT's u5 and vg that independent. Each small FL-PTT competes one large FL.-PT: (V1, 07), (12, vg), U5), and (14, 0g) are edges in the competing graph G.- Such Ge leads unique clique cover. Under this setting, small FL-PTs benefit large FL-PTs little. The experiments results (measured by mean squared eтror (MSE)) are given in Table
Strongly Non-IID setting, We generate confticting learning tasks by flipping over the labels of some FL-PTTs:
k=l T +c, 31,2,3,4)
(12)
y E
E o +e,i (5,6,7,8)
(13)
Different from the Weakly Non-IID setting, each FL-PT has 2000 samples, implying that there is no quantity skew. The setting of flipped labels between Eq. 122 and (33) leads to strongly Non-IID among the eight FL-PTs. We test on different competing graph where there are two independent groups of FL-PTTs [wa]1_ and J for E (1,5). the FL.-PTs and 0i+1 are independent and compete with 6 and 4/43 that are independent. Under this setting, all FL--PTs can benefit each other. The experiments results are given in Table
23

---

Table Experimental results (MSE) with synthetic data under fixed competing graphs: The weakly noD setting
U1
1/4
U/
LOCAL
0.32 + 0.05
0.28 = 0.00 1.00 + 0.07 0.69 + 0.08 0.28 # 0.02 0.28
0.01
0.72 = 0.0G 0.90 # 0.11
FEDAVE
0.25 1 0.01
0.25
1 0.01 0.79
0.05 0.55 0.05 0.23
0.01
0.23
1: 0.00
0.61
0.04
0.74 1 0.07
FEDPROX
0.26 - 0.01 0.27
0.01 0.90 0.10 0.67
0.00 0.26
0.76 0.11 1.02 + 0.17
SCAFFOLD
0.27 1 0.01 0.28 1 0.00 0.90 1: 0.03 0.67 0.00 0.25 0.00
0.26 0.01 0.72 1 0.09
0.92 1: 0.10
PFEDME
0.28 1 0.02 0.29 0.03 1.13 1: 0.55 0.86 0.58 0.33
0.33
0.74
0.02
0.82 1: 0.04
PEDIIN
0.35 = 0.07 0.31 = 0.05
0.91 0.07 0.61
0.33
0.31
0.09
0.90
PFEDGRAPH
0.26 1 0.01 0.27 1 0.01
0.90
1 0.04
0.67 0.08
0.26
0.00 0.26
1: 0.00
0.74 1 0.08
0.99 1 0.05
FEDEGOISTS 0.23 = 0.24 0.00 0.24 0.01 0.22 0.02
0.22 0.00 0.23 0.01
0.25 0.01 0.25 0.02
Table Experimental results (MSE) with synthetic data under fixed competing graphs: The strongly non-IID setting
P1
1/3
1/4
1/
04
LOCAL
0.29 + 0.03 0.29
0.02
0.20 + 0.00 0.29 0.04
0.27 0.01
0.27
0.04
0.27 = 0.02 0.27 + 0.01
FEDAVE
0.25 + 0.00
0.25
0.01 0.23
0.01
0.23
0.23
0.22
0.00
0.23 1 0.02 0.24 1 0.02
FEDPROX
0.27 + 0.02
0.00 0.26
0.01
0.20
0.24
0.24
0.25
0.01 0.25 0.01
SCAFFOLD
0.26 1 0.01 0.26
:: 0.01 0.26
:: 0.01 0.26
0.00
0.24
0:44:000
0.25
0.01
0.01 0.25 1 0.01
PFEDME
0.36 1 0.12 0.37 1 0.12 0.25
0.00 0.25 4:
0.28 0.02
0.27
0.01
0.27
0.01 0.28 1 0.01
PREDHN
0.33 + 0.05 0.34 = 0.02 0.28
0.34
0.29
0.05
0.29
PFEDGRAPH
0.26 1 0.01
0.27
1 0.01 0.26 1: 0.02
0.26
1 0.02
0:24
0.00
0.24 0.01
0.25
+ 0.01 0.25 1 0.01
FEDEGOISTS
0,24 + 0.00
0.27 0.05
0.24 0.03
0.22 0.01
0.22
0.22 0.00
0.22 0.01
0.22 0.01
B.7 Computer Resources
The system is equipped with an Intel(R) Xeon(R) Gold 6148 CPU operating at 2.40GHz. It utilizes 16 Nvidia Tesla V100 GPUs, each with 32GB of memory. The installed CUDA version 11.7, and the graphics driver version is 515.48.07.
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: As stated in the last part of the abstract and introduction, clearly summarized the contribution and scope of this paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: In the section 1 Introduction, we discuss in detail for improvement and limitations of the methods presented in this paper.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
25

---

Answer: [Yes]
Justification:
It can be seen from Section 3 Model and Assumptions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification:V We provide detailed experimental settings, datasets achieve reproducibility. See Section Evaluation.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
26

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: The code used in the experiments is openly accessible..
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips //iips.cc/pill wttp:ccppppcccceeeeeeeeee..c.e httae//ggiieee/eeeeebbsssssssss /Cc////oddiiisssseeeeeee ssiiiisiicyiiyyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: We have provided detailed experimental settings. See Section 5.1 Experimental Setup and Appendix B.3
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We provided the standard deviation of the results in the experimental results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
27

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
In the B.7 We have provided detailed experimental environment.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpps:/neurips.. http://cc..ppppiieccccccccceeeed
Answer:
[Yes]
Justification:
The research fully adheres to the NeurIPS Code in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work focuses on building FL ecosystem that can be applied to real-life business scenarios.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
28

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: When comparing with the baseline algorithm, its code used and this article was cited.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
29

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssit
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new data collection was done.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
No crowdsourcing conducted research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: No research with Human Subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
30

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31


Reviews:
[{"review_id": "fjLQJ7U2zd", "reviewer": "Reviewer_jdA3", "summary": "This paper focuses on client selection in cross-silo federated learning. The authors propose FedEgoists. In particular, FedEgoist participate clients into different clusters to avoid free riders and conflict of interests. Theoretical analysis is provided to validate the theoretical soundness of the proposed FedEgoist.\nExperiments validate the effectiveness of proposed FedEgoist.", "questions": "Please refer to weaknesses.", "limitations": "There is no potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The discussed topic, client selection in cross-silo federated learning, is important in practical setting.\n* The writing is easy to follow.", "weaknesses": "* The evaluation setting on CIFAR-10 and CIFAR-100 cannot simulate conflict of interests in the real world. Further consideration needed.\n* The meaning of propositions 1 & 2 is unclear. Further explanation is expected.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "The authors would like to thank you sincerely for your overall positive comments, including the acknowledgement of the importance of the discussed topic in practical setting and the theoretical soundness of the proposed framework. Your comments are also constructive to help improve the manuscript. Below, we made a response to the two questions in the weakness part, indexed as W1 & W2 respectively.\n\n\n**W1: Experiments on CIFAR-10 & CIFAR-100**\n\nIn this rebuttal, we better clarify the setting in our experiments. The setting here is the same as the setting in [30] where the same competitive relationships between clients are considered in the FL context. Two clients (i.e., FL-PTs in the paper) are either independent of each other, or compete against each other. For each pair of clients, there is a probability $\\alpha$ that these two clients compete; here the probability that these two clients are independent of each other is $1 - \\alpha$. The value of $\\alpha$ determines the intensity of competition among clients and a larger value reflects a higher level of competing intensity among clients.\n\nCompared with [30], we define a new metric to better validate the performance of the proposed approach and conducted more experiments to show the robustness of the proposed approaches in terms of the level of competing intensity between clients.\n\n+ Firstly, we show the performance of the proposed approach when $\\alpha$ takes different values in {0.05,0.1,0.2,0.3,0.4}, representing different levels of competing intensity between clients. We conducted five trials to show the average performance. For the $l$-th trial, a particular competing graph $G_{c,l}$ is randomly generated for the $l$-th time with the given $\\alpha$; then, the experiments for the baseline and proposed approaches are run; the performance of the proposed approach is denoted as $r_{\\alpha, l, p}$ while the performance of the $i$-th baseline approach is denoted as $r_{\\alpha, l, i}$. Given the value of $\\alpha$, we show the average performance of the five trials (i.e., $\\sum_{l=1}^{5}{r_{\\alpha, l, p}}/5$ and $\\sum_{l=1}^{5}{r_{\\alpha, l, i}}/5$, $i=1,\\dots, 9$). The related experimental results are presented in Tables 1 and 2. All tables can be found in the uploaded PDF document of the global rebuttal. For CIFAR-10, the proposed FedEgoists achieves the best performance in all cases when compared with all the basedline approaches. For CIFAR-100, the proposed FedEgoists achieves the best performance on average when compared with all the basedline approaches. \n\n+ Secondly, suppose we are given a specific value of $\\alpha$. We define a new metric to show the worst-case performance of the proposed approach compared with the baseline approaches by the following ways: \n   - Firstly, we find an integer $l^{\\ast}$ such that under the $l^{\\ast}$-th trial, the performance of the baseline approaches is the best compared with the proposed approach, i.e. $l^{\\ast} = \\arg\\max\\limits_{l\\in [1, 5]}\\{ (\\max\\limits_{i\\in [1,9]}{r_{\\alpha, l, i}}) - r_{\\alpha, l, p} \\}$ where $\\max\\limits_{i\\in [1,9]}{r_{\\alpha, l, i}}$ is the best performance of all the baseline approaches in the $l$-th trial and $(\\max\\limits_{i\\in [1,9]}{r_{\\alpha, l, i}}) - r_{\\alpha, l, p}$ is their performance improvement (or the difference) to the proposed approach, which may be negative if the proposed approach achieves a better performance. \n   - Secondly, after finding such $l^{\\ast}$, we show the performance of the proposed and baseline approaches under the $l^{\\ast}$-th trial. Specifically, given the value of $\\alpha$, we compute the value of $(\\max\\limits_{i\\in [1,9]}{r_{\\alpha, l, i}}) - r_{\\alpha, l, p}$; these values under different $\\alpha$ are presented in Table 3.\n\n$\\color{blue}{Conclusion.}$ Table 3 shows that in the worst case, the proposed FedEgoists has a performance very close to the best performance of all the baseline approaches. Tables 1 and 2 show that on average, the proposed FedEgoists achieves a significant performance improvement when compared with all the baseline approaches. \n\n\n**W2: The Meaning of Propositions 1 & 2**\n\nYou raised an excellent point about the clarity. In the final version, we will better clarify the meaning of Propositions 1 \\& 2 and better show how Algorithm 1 gives an optimal solution to the problem of this paper defined in Section 3.3 (lines 183-185 on page 5). \n\nSpecifically, clients are partitioned into multiple coalitions/groups, and this paper aims to find a solution that satisfies Principles 1 \\& 2 and $\\Pi=\\emptyset$. Principles 1 \\& 2 guarantees that there are no free-riders and conflicts of interests in a FL ecosystem; the requirement of $\\Pi=\\emptyset$ guarantees that no coalitions can collaborate together and be merged into a larger one to achieve a higher utility. In such context, the meaning of Propositions 1 \\& 2 is as follows:\n   + Proposition 1 shows that Eq. (1) holds and Principle 2 is realized in the solution given by Algorithm 1. Eq. (1) is given in Section 3.2.1 where we say that _Principle 1 is realized when Eq. (1) is satisfied_, which will be highlighted as a lemma in the final version and used in the proof of Proposition 1. Also, Proposition 1 will be updated to directly say that ``_Upon completion of Algorithm 1, Principles 1 \\& 2 are realized._\".\n   + Proposition 2 shows that the requirement of $\\Pi=\\emptyset$ in Eq. (3) is satisfied.\n\nIn the final version of the manuscript, at the end of Section 4, the physical meaning of the conclusions in Propositions 1 & 2 will be clarified better as explained above.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r53WPFo8td", "reviewer": "Reviewer_BcXD", "summary": "The business sector is a main domain where cross-silo federated learning (FL) has many promising applications in various scenarios. The authors simultaneously consider the self-interest and competition features in the business sector. They develop a novel framework to both address the resulting free-riding problem and avoid the conflict of interest between any two competing FL participants (FL-PTs), which results in an interesting optimization problem. Finally, the authors find an optimal solution where one coalition cannot increase the utility of any of its members by collaborating with other coalitions. Extensive experiments are conducted to show the effectiveness of the proposed solution, and its ability to establish efficient collaborative networks in cross-silo FL with FL-PTs that engage in business activities.", "questions": "Q1: Can the proposed technical framework be applied to other types of tasks or datasets, e.g., NLP? In the experimental evaluation part, the authors have already shown its application to some interesting tasks commonly seen in literature.\nQ2: To keep consistency, it is better that “selfish” in Figure 1 is changed to “self-interest”.\nQ3: In this paper, all FL-PTs are partitioned into mutually disjoint coalitions/groups. It seems that this is similar to clustered federated learning where FL-PTs are partitioned into mutually disjoint clusters (i.e., groups). The reviewer understands that the authors consider a problem that is clearly different from the typical clustered federated learning. \nQ4: In your experimental results, how many trials have you conducted to obtain the standard deviation?", "limitations": "See weaknesses and Questions.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "S1: The authors consider a timely and interesting question in cross-silo FL where the free-riding and competition issues are considered simultaneously. \nS2: The authors propose a new problem formulation and definition and develop a novel theoretical framework of practical importance to address the identified problem. The resulting optimization problem is solved optimally. The paper is technically sound.\nS3: The paper is well organized and easy to follow. The background and related work are well introduced to understand the importance of the problem of this paper. \nS4: Extensive experiments have been done to show the effectiveness of the proposed solution.", "weaknesses": "W1: The proposed algorithm is not polynomial-time solvable. However, this seems reasonable since there are typically a limited number of FL-PTs (e.g., 2 to 100) for cross-silo FL in business sectors. The authors propose a novel application of the classic algorithms in graph theory to solve their problem optimally. The algorithm complexity depends on these classic graph algorithms, which work well in reality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "The authors would like to thank you sincerely for your overall positive comments on the manuscript, including your positive acknowledgement of the question under study, the theoretical soundness of the proposed framework, and the effective experimental validation of the proposed solution. \n\n**Q1:** The method itself is general and is expected to be applicable to NLP field, which will be shown in a future journal version.\n\n**Q2:** In the final version, we will update the manuscript as you suggested.\n\n**Q3:** In the final version, we will better clarify in Section 2 the connection and differences of our work with clustered federated learning.\n\n**Q4:** In the experiments, five trials are conducted to obtain the standard deviation. We will clarify this in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9T46mjYVZz", "reviewer": "Reviewer_gR3Y", "summary": "This paper studies an interesting topic, which is about cooperation and competition in federated learning. It can be used to describe or simulate the real-world scenarios. The authors propose to use graph-related techniques to formulate the relation between the local clients. They test the algorithm on the benchmark datasets and the medical dataset.", "questions": "1, Could you please address the concerns in the weakness part?\n\n2, Could you clarify how the local model information is leveraged and updated in the graph part more clearly?\n\n3, How to validate or verify the meaning of the graph construction part? Do we need to have the groundtruth of the relation of all the clients?", "limitations": "1, The connection of the proposed techniques should be closely associated with the setting of FL, which focuses on the iterative updates between the server and the clients.\n\n2, The original novelty of contribution should be emphasized a little bit more.\n\n3, More comprehensive experiments may help demonstrate the effectiveness and understand the generalization of the algorithm better.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1, The research question in federated learning may have potential practical impact in the real-world setting. The cross-silo setting fits what the authors explore. Considering the cooperation and competition from the graph perspective in FL is an interesting point.\n\n2, Statements are supported with experiment results and theory.\n\n3, The presentation is helpful for readers including the algorithm description, diagram, and figures.", "weaknesses": "1, The novelty of each module, especially the graph part, may need to be clarified clearly. Also, the two principles are commonly used, if the reviewer understand correctly, an optimal formation is proposed based on these two, is that correct?\n\n2, Most designed part is at the server side. The reviewer may not be able to observe the strong connection with federated learning. The setting itself may tell that each client is a participant in this framework. However, the introduction (motivation) and the method could be a little bit fragmented.\n\n3, Ablation study and hyperparameter study may help verify the algorithm more deeply and soundly.\n\n4, A little bit concerned about the baseline selection. Some contribution-related work in FL like [1], if comparable, should be considered or at least discussed.\n\n[1] Contribution-Aware Federated Learning for Smart Healthcare", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Weaknesses 1 \\& Limitations 2** \n\t\nOverall, the original novelty of contribution includes multiple aspects: (1) identifying an interesting question to study, (2) proposing a desirable solution concept for this new problem, and (3) proposing an optimal solution. \n\nSpecifically, business sectors are a key FL application domain. Firstly, the concerns of FL participants are identified (Section 1). Although the principles used to address these concerns are intuitive or used in FL literature, they are simultaneously considered for the first time. Secondly, a solution concept (or a proper problem formulation) is proposed (Section 3.3) such that the resulting solution can well satisfy the needs of FL participants and help FL participants achieve the best possible ML model performances. Thirdly, an algorithm is proposed to give an optimal solution to the defined problem (Section 4).  \nThe aim of developing FL techniques is to make FL able to be implemented in real worlds and adopted by organizations. The proposed theoretical framework of this paper serves as a starting point to construct a FL ecosystem where FL-PTs have both self-interest and competition features and a FL manager needs to construct collaboration relationships of FL-PTs without free-riding and conflicts of interests between FL-PTs.\n\nYes, the solution concept of this paper is proposed based on these two principles. Of the greatest relevance to this paper is Ref. [30] that only considers Principle 1 in this paper and simply gives a heuristic solution to realize Principle 1. In this paper, we are motivated by the scenario under study and consider simultaneously realizing Principle 1 as well as Principle 2. Also, a refined solution concept is proposed in Section 3.3 to guarantee that no coalitions can collaborate together and be merged into a larger one to achieve a higher utility. \n\n**Weaknesses 2 \\& Limitations 1** \n\nThis paper is about the application of federated learning (FL) to business sectors and addresses the related issues about free-riders and conflicts of interests, which are the key to enabling the application of FL in the scenario under study. It is interdisciplinary in nature and may relate to the topics “Applications” and “Social and economic aspects of machine learning” in the Call for Papers of NeurIPS-2024. Its focus is not on the FL’s technical details or how to improve the potential technical flaws in the current FL study; in contrast, its focus is on the social and economic effect of FL. Applications are the destinations of a technique; business sectors are an important domain to apply FL; addressing better the concerns of FL participants in business sectors can facilitate the application of FL, thus amplifying its social and economic effect. On the other hand, in order to validate the effectiveness of the proposed theoretical framework, related FL techniques have to be implemented in the experimental evaluation part (i.e., Section 5). \n\n**Weaknesses 3** \n\nMotivated by your suggestions, more experiments have been conducted to verify the algorithm better. \n\nThe experiments aim to verify the effectiveness of the proposed theoretical framework in the scenario under study. The scenario features self-interest for every client and competition between a part of clients. The collaboration relationships between clients are built according to the competing graph and the benefit graph; the latter graph depends on the data heterogeneity/complementarity of clients. Thus, the variable factors in the scenario/experimental environment include the intensity of competition between clients and the non-IID setting. Following your suggestions, we vary the related parameters in these two factors to verify the algorithm better.\n\nFirstly, two clients are either independent of each other, or compete against each other. For each pair of clients, there is a probability $\\alpha$ that these two clients compete; here the probability that these two clients are independent of each other is $1 - \\alpha$. The value of $\\alpha$ determines the intensity of competition among clients and a larger value reflects a higher level of competing intensity among clients. In the benchmark experiments on CIFAR-10 and CIFAR-100, we better verify the effect of the competition intensity on the effectiveness of the proposed solution where we vary the value of $\\alpha$ that takes different values in {0.05,0.1,0.2,0.3,0.4} and conducted the corresponding experiments. The related experimental results are presented in Tables 1 & 2. All the tables can be found in the PDF document of the global rebuttal. \n\nSecondly, the data heterogeneity is typically simulated by a pathological or Dirichlet distribution. e.g., the pathological distribution is used in [5,30]. In this paper, we conducted experiments on pathological and Dirichlet distributions, respectively. Also, in this rebuttal, we better verify the effect of data heterogeneity on the effectiveness of the proposed solution. Specifically, let $m$ denote the number of classes. In Dirichlet distribution, there is a distribution vector $q_{c}\\in \\mathbb{R}^{m}$ is drawn from the Dirichlet distribution $Dir_{m}(\\beta)$ for each class $c$ and client $v_{i}$ is allocated a $q_{c,i}$ proportion of data samples of class $c$; smaller $\\beta$ value results in higher data heterogeneity. We vary the value of $\\beta$ that takes different values in {0.01,0.1,0.5} like [39] and conducted the corresponding experiments. The related experimental results are presented in Table 4.\n\nBesides the added experiments, we better clarify the type or the research focus of this paper. In the experiments, since this paper doesn’t focus on researching the FL technique itself, we simply follow the current best technical practices in FL literature. Like [5,30], ablation study and hyperparameter study are not taken and this paper doesn’t focus on the design of network architectures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cBFY3Gcdrf", "reviewer": "Reviewer_2xqy", "summary": "This paper introduces a strategy named FedEgoists designed to enhance collaboration in cross-silo federated learning (FL) scenarios, particularly in business sectors where participants (FL-PTs) are often competitive and self-interested. The proposed FedEgoists strategy presents a sophisticated and theoretically sound approach to managing collaborations in cross-silo federated learning, and its strengths lie in its ability to handle self-interest and competition effectively.", "questions": "(1) FedEgoists requires a central server to coordinate and enforce the coalition formation, which introduces a single point of control and potential failure. This centralization might conflict with the decentralized nature of federated learning and violate the fundamental motivation of FL.\n(2) FedEgoists assumes that FL-PTs will accurately report their competitive relationships to the central server (CS). However, this can easily lead to privacy leaks and potential attacks.\n(3) While the FedEgoists strategy focuses on collaboration benefits and competition avoidance, it does not explicitly address data complementarity.\n(4) Since the FedEgoists strategy relies on coalition formation based on reported benefits and competition, there is a risk of strategic manipulation by participants.\n(5) The experiments are not sufficient, and there are too few datasets and baselines.", "limitations": "It seems limitations are not clearly or explictly elaborated.", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "FedEgoists ensures that the formed coalitions are optimal. No coalition can improve its utility by merging with another coalition, making the solution stable and efficient. The strategy effectively addresses the problem of free riders, ensuring that all FL-PTs contribute to and benefit from the FL ecosystem proportionately. By preventing FL-PTs from contributing to their competitors or their supporters, the strategy minimizes conflicts of interest. This paper validates FedEgoists using the benchmark experimental datasets.", "weaknesses": "The paper lacks rigorness as pointed out in the Questions part.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Comments 1.** FedEgoists requires a central server to coordinate and enforce the coalition formation, which introduces a single point of control and potential failure. This centralization might conflict with the decentralized nature of federated learning and violate the fundamental motivation of FL. Since the FedEgoists strategy relies on coalition formation based on reported benefits and competition, there is a risk of strategic manipulation by participants.\n\n\n$\\color{blue}{Response.}$ In the classic federated learning (FL) framework, there is a central server that is assumed to be trustable. The decentralized nature of FL refers to the data of different FL-PTs (i.e., FL participants) being decentralized since the local data of FL-PTs don’t need to be transferred to the central server.\n\nAs introduced in Section 1, in the vanilla FedAvg framework, multiple FL-PTs train a shared model locally with their own dataset, and upload their local model updates to a central server trusted by all FL-PTs, which then aggregates these model updates and distributes the model updates to each FL-PT (i.e., client) [22]. Following such a client-server architecture of FedAvg, many important techniques have been proposed to improve the FL performance while facing the statistical heterogeneity of data over FL-PTs (e.g., FedProx [18], SCAFFOLD [14], pFedHN [26], pFedMe [28], FedDisco [40], pFedGraph [39]). Such client-server architectures are especially applicable to cross-silo FL considered in this paper; here, FL-PTs are typically companies or organizations with reliable computational resources and communication channels [9,10]. In such architectures, the central server has authority to determine the contribution relationships of FL-PTs (i.e., the ways of aggregating the model updates for FL-PTs) in the FL training process (e.g., [5,6,30]).\n\nThere is a development process for a technique to be implemented in real worlds and adopted by organizations. Business sectors are an important FL application domain; addressing better the concerns of FL participants in business sectors can facilitate the participation of potential organizations into a FL ecosystem. The theoretical framework of this paper sticks to the current FL technical practices and serves as a starting point to construct a FL ecosystem where FL-PTs have both self-interest and competition features and a FL manager needs to construct collaboration relationships of FL-PTs without free-riding and conflicts of interests between FL-PTs.\n\nThe reviewer also raised an excellent point about a single point of control and potential failure. In some important application domains of FL such as wireless network systems, it is necessary to consider such issues with a single point of failure; as shown by the survey below, in such domains, blockchain-based federated learning can be promising where there is no central server and peer-to-peer coordination between FL-PTs is needed:\n\n- Nguyen, Dinh C., Ming Ding, Quoc-Viet Pham, Pubudu N. Pathirana, Long Bao Le, Aruna Seneviratne, Jun Li, Dusit Niyato, and H. Vincent Poor. \"Federated learning meets blockchain in edge computing: Opportunities and challenges.\" *IEEE Internet of Things Journal* 8, no. 16 (2021): 12806-12825.\n\nIn blockchain-based FL without a central server, it is an interesting research problem to design proper negotiation protocols to guarantee that there are no free-riding and conflicts of interests between FL-PTs. This will be highlighted as a nice future work in the final version of our manuscript.\n\n**Comments 2.** FedEgoists assumes that FL-PTs will accurately report their competitive relationships to the central server (CS). However, this can easily lead to privacy leaks and potential attacks.\n\n\n**$\\color{blue}{Response.}$** Following the standard technical practices in literature, the central server is assumed to be trustable, as explained above. In real worlds, the central server may represent an impartial and authoritative third-party (e.g., the industry association) [5]. Then, FL-PTs can report their competitive relationships to the third-party in person and confidentiality agreements can be signed between the third-party and FL-PTs. We will clarify this in the final version of the manuscript.\n\n**Comments 3.** While the FedEgoists strategy focuses on collaboration benefits and competition avoidance, it does not explicitly address data complementarity.\n\n**$\\color{blue}{Response.}$** Yes, this paper doesn’t address the technical issues related to data complementarity. It uses the existing techniques [5,30] to evaluate the data complementarity between FL-PTs, which is represented by a benefit graph. The competitive relationships between FL-PTs form a competing graph. According to the benefit graph and the competing graph, this paper aims to construct the contribution relationships between FL-PTs to guarantee no free-riding and conflicts of interests between FL-PTs, which is desired in business sectors.\n\n**Comment 4.** The experiments are not sufficient, and there are too few datasets and baselines.\n\n**$\\color{blue}{Response.}$**  Due to the space limitations, please refer to global rebuttal.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
More experiments have been conducted to verify the effectiveness of the proposed solution:
1. More experiments are conducted to verify the robustness of the proposed algorithm.
2. We define a new metric to better validate the performance of the proposed approach.
3. New datasets and baselines are considered.

**Added Experiments Part 1: Robustness**

The paper is about the application of the current FL techniques to the business sectors, and the experiments aim to verify the effectiveness of the proposed theoretical framework in the scenario under study. The scenario features self-interest for every client and competition between a part of clients. The collaboration relationships between clients are built according to the competing graph and the benefit graph; the latter graph depends on the data heterogeneity or complementarity of clients. Thus, the variable factors in the scenario or experimental environment include the intensity of competition between clients and the non-IID setting. We vary the related parameters in these two factors to verify the robustness of the proposed algorithm.

Below, we explain the related parameters in these two factors and the added experiments:

Firstly, two clients are either independent of each other or compete against each other. For each pair of clients, there is a probability $\alpha$ that these two clients compete; here the probability that these two clients are independent of each other is $1 - \alpha$. The value of $\alpha$ determines the intensity of competition among clients and a larger value reflects a higher level of competing intensity among clients. In the benchmark experiments on CIFAR-10 and CIFAR-100, we better verify the effect of the competition intensity on the effectiveness of the proposed solution where we vary the value of $\alpha$ that takes different values in {0.05,0.1,0.2,0.3,0.4} and conducted the corresponding experiments.Given a specific value of $\alpha$, we conducted five trials to show the average performance. For the $l$-th trial, a particular competing graph $G_{c,l}$ is randomly generated for the $l$-th time with the given $\alpha$; then, the experiments for the baseline and proposed approaches are run; the performance of the proposed approach is denoted as $r_{\alpha, l, p}$ while the performance of the $i$-th baseline approach is denoted as $r_{\alpha, l, i}$. Given the value of $\alpha$, we show the average performance of the five trials (i.e., $\sum_{l=1}^{5}{r_{\alpha, l, p}}/5$ and $\sum_{l=1}^{5}{r_{\alpha, l, i}}/5$, $i=1,\dots, 9$).
The related experimental results are presented in Tables 1 and 2.

Secondly, the data heterogeneity is typically simulated by a pathological or Dirichlet distribution. For example, the pathological distribution is used in [5,30]. In this paper, we conducted experiments on pathological distribution(PAT.) and Dirichlet distribution(Dir), respectively. Also, in this rebuttal, we better verify the effect of data heterogeneity on the effectiveness of the proposed solution. Specifically, let $m$ denote the number of classes. In Dirichlet distribution, there is a distribution vector $q_{c}\in \mathbb{R}^{m}$ is drawn from the Dirichlet distribution $Dir_{m}(\beta)$ for each class $c$ and client $v_{i}$ is allocated a $q_{c,i}$ proportion of data samples of class $c$; smaller $\beta$ value results in higher data heterogeneity. We vary the value of $\beta$ that takes different values in {0.01,0.1,0.5} like [39] and conduct the corresponding experiments. The related experimental results are presented in Table 4.

**Added Experiments Part 2: A New Metric**

Suppose we are given a specific value of $\alpha$. We define a new metric to show the worst-case performance of the proposed approach compared with the baseline approaches by the following ways:
1. We find an integer $l^{\ast}$ such that under the $l^{\ast}$-th trial, the performance of the baseline approaches is the best compared with the proposed approach, i.e. $l^* = \arg \max_{l \in [1,5]} \left( \max_{i \in [1,9]} r_{\alpha,l,i} - r_{\alpha,l,p} \right)
$ where $\max\limits_{i\in [1,9]}{r_{\alpha, l, i}}$ is the best performance of all the baseline approaches in the $l$-th trial and $(\max\limits_{i\in [1,9]}{r_{\alpha, l, i}}) - r_{\alpha, l, p}$ is their performance improvement (or the difference) to the proposed approach, which may be negative if the proposed approach achieves a better performance.
2. After finding such $l^{\ast}$, we show the performance of the proposed and baseline approaches under the $l^{\ast}$-th trial. Specifically, given the value of $\alpha$, we compute the value of $(\max\limits_{i\in [1,9]}{r_{\alpha, l, i}}) - r_{\alpha, l, p}$; these values under different $\alpha$ are presented in Table 3.

$\color{blue}{Conclusion.}$ Table 3 shows that in the worst case, the proposed FedEgoists has a performance very close to the best performance of all the baseline approaches. Tables 1 and 2 show that on average,  FedEgoists achieves a significant performance improvement when compared with all the baseline approaches. 


**Added Experiments Part 3: A New Dataset and a New Baseline**

More experiments are conducted on a new dataset and using a new baseline. Specifically, we additionally conducted experiments on the synthetic data used in [30] where the experimental setting is also the same as the setting in [30]. The related experimental results are presented in Tables 5-6. We add one more approach in the paper below, called FEDORA, as the baseline:
- Jun Wu, Wenxuan Bao, Elizabeth Ainsworth, and Jingrui He. "Personalized federated learning with parameter propagation." In *ACM KDD*, 2023.
FEDORA is only for image classification tasks; thus, we only conducted experiments on CIFAR-10 and CIFAR-100. The related experimental results are presented in Tables 1,2,4.

$\color{blue}{Remark.}$  All experimental results (tables) can be obtained from the uploaded PDF document. The results are presented in the form of mean ± std.

Meta-review: In this paper, the authors focus on the selection of federated learning participants and study the problem of free riders and conflicts of interest among competitors. Now, the paper has received four reviews: one Accept, two Weak Accept, and one Borderline Reject.

From the positive points of the reviews, the reviewers have generally agreed that (1) the paper has studied an important problem in federated learning, (2) there are some interesting results, and (3) the paper is easy to follow. However, at the same time, the reviewers have pointed out several weaknesses on the evaluation, complexity, novelty, motivation, etc.

During the rebuttal, the authors have provided feedback to each reviewer on these weaknesses, and most of the reviewers remain positive after reading the rebuttal. Moreover, the rebuttal has clarified quite a few concerns raised by the reviewers. Since the overall scores are positive and above the acceptance threshold, we recommend accepting this paper for NeurIPS 2024.
Decision: Accept (poster)