Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learning

Authors: mengmeng  chen
xiaohu  wu
xiaoli  tang
tiantian  he
yew-soon  ong
qiqi  liu
qicheng  lao
han  yu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 14:08:16

Abstract: Federated learning (FL) is a machine learning paradigm that allows multiple FL participants (FL-PTs) to collaborate on training models without sharing private data. Due to data heterogeneity, negative transfer may occur in the FL training process. This necessitates FL-PT selection based on their data complementarity. In cross-silo FL, organizations that engage in business activities are key sources of FL-PTs. The resulting FL ecosystem has two features: (i) self-interest, and (ii) competition among FL-PTs. This requires the desirable FL-PT selection strategy to simultaneously mitigate the problems of free riders and conflicts of interest among competitors. To this end, we propose an optimal FL collaboration formation strategy -FedEgoists- which ensures that: (1) a FL-PT can benefit from FL if and only if it benefits the FL ecosystem, and (2) a FL-PT will not contribute to its competitors or their supporters. It provides an efficient clustering solution to group FL-PTs into coalitions, ensuring that within each coalition, FL-PTs share the same interest. We theoretically prove that the FL-PT coalitions formed are optimal since no coalitions can collaborate together to improve the utility of any of their members. Extensive experiments on widely adopted benchmark datasets demonstrate the effectiveness of FedEgoists compared to nine state-of-the-art baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.

Main paper: 
 introduction: 
 Federated learning (FL) is a promising paradigm of distributed machine learning (ML) as it does not require sharing raw data between FL participants (FL-PTs), thereby upholding the privacy considerations [11, 13, 55, 56, 57] . In the popular Federated Averaging (FedAvg) framework, multiple FL-PTs train a shared model locally with their own datasets, and upload their local model updates to a central server (CS), which then aggregates these model updates and distributes the resulting global model to each FL-PT [33] . There are two types of FL [21] . In cross-device FL, FL-PTs are end-user devices such as smartphones or IoT devices, and CS is the final owner of the trained model. In cross-silo FL, FL-PTs are companies or organizations in private or public sectors and are the final owners/users of the trained model, while the CS has the authority to coordinate the FL training process.There exists competitions between companies.√Ensures the absence of the free riders; √Avoids the conflict of interest; √Optimal coalition formation with the two constraints above.Figure 1 : An overview of the main motivation and results of this paper.We consider in this paper the scenario of cross-silo FL where organizations in the business sector that engage in business activities are key sources of FL-PTs [17] . The business sector is part of the private sector made up by companies and includes business that operate for profit; it is a key driver of technological innovation. The application of FL in the business sector has been studied in diverse domains, including digital banking, ridesharing, recommender systems, and Electric Vehicle charging services [29, 30, 40, 48, 54] . Regional banks have different user groups from their respective regions and are independent [55] , while the banks in the same region can compete for users [32] . Recently, a FL platform, MELLODDY, has been developed for drug discovery, including multiple companies from business sectors [36] . Generally, competition exists when there are multiple organizations that are in the same market area and hope to unlock the power of FL [17, 46] .From a FL-PT's perspective, the scenario under study has two features: (i) self-interest and (ii) competition among FL-PTs. In business sectors, individuals are self-interested. This magnifies the free-riding problem where some FL-PTs benefit from the contributions of others without making any contribution to the FL ecosystem [21, 23] . Competition signifies that there is a potential conflict of interest between some two FL-PTs. Thus, two principles can simultaneously be used to meet the individual's needs: (1) a FL-PT can benefit from the FL ecosystem if and only if it can benefit the FL ecosystem, and (2) a FL-PT will not contribute to its competitors as well as the allies of its competitors [43] . The first principle is proposed and formulated in this paper for the first time, while the second principle has previously been considered for other scenarios [43] . From a FL manager's perspective, its priority is to regulate the FL ecosystem to meet the requirements of its customers (i.e., FL-PTs).This paper studies cross-silo FL in business sectors where there are typically a limited number of FL-PTs (e.g., 2 to 100) [17, 28] . We consider both self-interest and competition for the first time in literature, although it has been well recognized that the free riding problem can be especially serious when self-interested FL-PTs engage in business competition [18, 42] . Since competition and self-interest are considered, all FL-PTs are partitioned into disjoint groups/coalitions, each with the common interest. The competition and benefit relationships among FL-PTs can be described as graphs. We use theoretical tools from graph theory and propose an efficient solution FedEgoists that can well satisfy the two principles above. Meanwhile, FedEgoists can help FL-PTs achieve the best possible ML model performances, i.e., subject to the two principles, the coalitions that FedEgoists finds are optimal in the sense that one coalition cannot increase the utility of any of its members by collaborating with any other coalitions. Extensive experiments over real-world datasets have demonstrated the effectiveness of the proposed solution compared to nine baseline methods, and its ability to establish efficient collaborative networks in cross-silo FL with FL-PTs that engage in business activities. The main motivation and results of this paper are illustrated in Figure 1 . 
related work: 
 The existing FL works consider the following three lines of works separately and don't address the self-interest and competition features simultaneously.Competition. Collaboration between competing companies is an important research area in business studies [3, 12] . Recently, there has been a growing interest in the study of competition in cross-silo FL where a key application area is the business sector. There are two cases. In the first case, all FL-PTs are assumed to offer the same service in the same market area and compete against each other. Several works attempt to reduce the potential side effect after FL-PTs join a fully competitive FL ecosystem. Wu and Yu [51] aim to maintain a negligible change in market share [52, 53] and analyze the achievability of this objective. Tsoy and Konstantinov [46] and Huang et al. [17] study the profitability of FL-PTs, but are taken under different assumptions on the source of extra profit brought by FL. Specifically, Tsoy and Konstantinov [46] assume that: (i) each consumer allocates a fixed budget to multiple services from different markets, and (ii) if a FL-PT owns a better model, the consumer will thus allocate more of its budget to consume its service due to the improved service quality. Huang et al. [17] consider duopoly business competition and assume that, if the model-related service can be improved by FL, customers will have a willingness to pay more and FL-PTs thus have opportunities to increase their profits. In the second case, not all FL-PTs compete against each other, like one assumption in this paper, where some FL-PTs come from different market areas. Tan et al. [43] study how to avoid the conflicts of interest among FL-PTs for the first time and give a heuristic to realize Principle 2. However, Tan et al. [43] don't consider the self-interest feature of cross-silo FL in the business sector and cannot guarantee the optimal collaboration among independent coalitions without competition. The collaboration relationships among FL-PTs result in a collaboration graph G u . In their heuristic, FL-PTs are considered one by one and G u is thus constructed gradually to guarantee that for each FL-PT, there is no path that can be reachable to or from its competitors. In this paper, we apply the concepts of cliques and strongly connected components in graph theory to satisfy the self-interest and competition features and further optimize the proposed algorithm.Free-riding. Karimireddy et al. [23] show that a naive scheme can lead to catastrophic levels of freeriding where the benefits of data sharing are completely eroded. By contract theory, they introduce accuracy shaping based mechanisms to prevent free-riding.Coalitions. A coalition is a cooperative relationship formed among different individuals due to certain interests. Each agent is only concerned with the contributions of other agents within the same coalition [1] . Prior studies mainly focus on alleviating the side effect of data heterogeneity by allocating proper collaborators with data complementarity to each FL-PT. Donahue and Kleinberg [10] provide an analytical understanding of what partition of FL-PTs leads to a core-stable coalition structure for mean estimation and linear regression. Cui et al. [5] propose a heuristic algorithm to compute the core-stable coalition structure for general learning tasks. To some extent, the CS can dictate the collaboration relationships among FL-PTs only if doing so can better address the concerns of FL-PTs such as fairness and performance [7, 14, 60, 61] . Chaudhury et al. [4] treat all FL-PTs as a grand coalition and optimize a common model such that there is no other coalition S of FL-PTs that could significantly benefit more by training a model only using the data of S. Sattler et al. [38] propose a novel federated multi-task learning framework where the geometric properties of the FL loss surface is leveraged to group FL-PTs into clusters with jointly trainable data distributions. Ding and Wang [8] partition a total of n FL-PTs into K groups, where K ≪ n and the FL-PTs with similar contributors are assigned to the same group. Then, a common model is returned to each group of FL-PTs such that the performance of using K models for n FL-PTs is a good approximation to the the performance of learning n personalized models for n FL-PTs respectively. However, the above works don't consider the adversarial relationships among FL-PTs.3 Model and AssumptionsQuestion: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: In the section 1 Introduction, we discuss in detail for improvement and limitations of the methods presented in this paper.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
relationship graphs: 
 There is a set of FL-PTs denoted by )) and is used to evaluate the data complementarity between FL-PTs. It is defined as follows. For any two FL-PTs v j and v i , if v i can benefit from v j 's data, then there is a directed edge from v j to v i (i.e., (v j , v i ) ∈ E b ) and the weight of this edge is w j,i > 0 where a larger value of w j,i signifies a larger benefit to v i brought by v j . In contrast, if v i cannot benefit from the data of v j , then (v j , v i ) / ∈ E b and w j,i = 0. The benefit graph can be computed by the hypernetwork technique like [5, 34, 43] . The competing graph G c is an undirected graph with an edge set E c , i.e., G c = (V, E c ). For any two FL-PTs v i and v j , if they compete against each other, then there is an undirected edge between v i to v j (i.e., (v i , v j ) ∈ E c ). If they are independent of each other, then (v i , v j ) / ∈ E c . The CS is assumed to be trustable. In the real world, the CS may represent an impartial and authoritative third-party (e.g., the industry association) [5] . Then, FL-PTs can report their competitive relationships to the third-party in person and confidentiality agreements can be signed between the third-party and FL-PTs. Each FL-PT v i will report its competitors to CS, as it hopes that CS will correctly utilize this information to prevent its competitors from benefiting from its data. Thus, CS has the knowledge of G c . Although v i may benefit from v j 's data(w j,i > 0), CS has the authority to determine whether v i can actually utilize v j 's local model update information (i.e., indirectly use v j 's data) in the FL training process or not. Let X = (x j,i ) be a n × n matrix where x j,i ∈ {0, 1}: for two different FL-PTs v i and v j , x j,i is set to one if v j will contribute to v i (i.e., v i will utilize v j 's local model update information) in the FL training process and x j,i is set to zero otherwise. X defines a directed graph G u = (V, E u ), called the data usage graph: (v j , v i ) ∈ E u if and only if j ̸ = i, x j,i = 1; then, v j is said to be a collaborator or contributor of v i .V = {v 1 , v 2 , • • • , v n }. Each FL-PT v i isFinally, we introduce some concepts in graph theory. Let G = (V, E) denote an arbitrary graph whose node set is V and whose edge set is E. For any subset S ⊆ V, an (induced) subgraph G (S) of the graph G is such that (i) the node set of G (S) is V and (ii) the edge set of G (S) consists of all of the edges in E that have both endpoints in S. A subgraph G (S) is said to be a strongly connected component of G if we have (i) G (S) is strongly connected, i.e., there is a path in each direction between any two nodes and (ii) G (S) is maximal in the sense that no additional edges or nodes from G can be included in the subgraph without breaking the property of being strongly connected. The collection of strongly connected components forms a partition of the nodes of G. A simple path is a path in a graph which does not have repeating nodes.The FL-PTs in the scenario under study have two features: (i) self-interest and (ii) competition among FL-PTs. Accordingly, there are two collaboration principles introduced below.While establishing collaboration relationships among FL-PTs, the following principle is used to guarantee the non-existence of free rider. Principle 1. For any FL-PT v i ∈ V, there exists a FL-PT v j ∈ V that benefits v i if and only if there exists at least one FL-PT v k that can benefit from v i .In this paper, all Fl-PTs are assumed to be self-interested and there exists competition among some FL-PTs. Let π denote a partition of FL-PTs V into K mutually disjoint groups where πEQUATION, ..., K}. For any S k ∈ π, the collaboration relationships among the FL-PTs of S k are established according to the subgraph G b (S k ) , i.e., for any v i , v j ∈ S k , v j is a contributor of v i if and only if there exists an edgeEQUATIONIn a coalition S k with |S k | ⩾ 2, each FL-PT v i ∈ S k can both benefit and benefit from other members in S k -{v i }. Thus, in a set π of coalitions, Principle 1 will be realized since Eq. ( 1) is satisfied. If |S k | = 1, the single FL-PT of S k neither benefit nor benefit from any other FL-PT.To avoid conflict of interest, a FL-PT will not contribute to its competitors (i.e., its enemies) and any FL-PTs that help its competitors (i.e., the friends of its enemies). Also, it doesn't hope to see others help the supporters/friends of its competitors (in)directly. For any FL-PT v i , let A s denote the supporter alliance of v i defined as:A i = {v k ∈ V | v k is reachable to v i in the data usage graph G u }.Let us consider any two competing FL-PTs v j and v i where (v j , v i ) ∈ E c . To avoid conflict of interest, v j will not contribute to any member of A i . Like [43] , this is defined as the following principle by which it is strictly guaranteed that no FL-PTs will contribute to its competitors (in)directly. Principle 2. For any two competing FL-PTs v i and v j , v j is unreachable to v i in the data usage graph G u .Now, we propose a proper problem formulation such that the resulting solution can well satisfy the FL-PTs' needs and help them achieve the best possible ML model performances. All FL-PTs are divided into a set π of coalitions. For the FL-PTs of of a coalition S k ∈ π, their collaboration relationships are established according to the subgraph G b (S k ). The utility u(S k ) of a coalition S k is defined as the sum of the edge weights ofG b (S k ). Formally, let E b (S k ) denotes the edge set of G b (S k ); then, u(S k ) = (vj ,vi)∈E b (S k ) w j,i ; given a FL-PT v i ∈ S k , its utility is defined as u i (S k ) = (vj ,vi)∈E b (S k ) w j,iwhere v i is fixed; here, the utility of a coalition is the sum of the utilities of all its members, i.e., u(S k ) = vi∈S k u i (S k ). For two coalitions S k and S ′ k withS k ⊊ S ′ k , G b (S k ) is a subgraph of G b (S ′ k ); then the utility of v i in a larger coalition S ′ k is no smaller than its utility in a smaller coalition S k , i.e., u i (S k ) ⩽ u i (S ′ k ). In this paper, our final problem is to find a partition π of FL-PTs such that • Principles 1 and 2 are satisfied.• Subject to Principles 1 and 2, no coalitions of π (i.e., no subset π ′ of π) can collaborate together and be merged into a larger coalition with a higher utility; in other words, after merging these coalitions into a larger one, no FL-PTs in these coalitions π ′ increase their utilities under the larger coalition. Formally, letΠ = π ′ ⊆ π | S k ∈π ′ u(S k ) < u S k ∈π ′S k , Principles 1 and 2 are satisfied byS k ∈π ′ S k .Then, Π satisfies:EQUATION4 SolutionNow, we give an algorithm that satisfies Principles 1 and 2 and Eq. ( 2). It is presented as Algorithm 1 and its main idea is as follows. Firstly, we find a partition π = { Ŝ1 , Ŝ2 , • • • , ŜH } of all FL-PTs V such that the FL-PTs of each subset Ŝh ∈ π are independent of each other. Secondly, Ŝh ∈ π is further partitioned into several subsets/coalitions, denoted asSCC h = { Ŝh,1 , Ŝh,2 , • • • , Ŝh,y h } such that for all l ∈ [1, y h ], G b ( Ŝh,l ) is a strongly connected component of G b ( Ŝh ).Thirdly, for any coalitions of H h=1 SCC h , we merge these coalitions into a larger one if doing so achieves a higher coalition utility without violating Principles 1 and 2.Below, we detail Algorithm 1. Firstly, we define G - c = (V, E - c ) as the inverse of the competing graph G c , i.e., any two nodes of G - c are adjacent if and only if they are not adjacent in G c ; if there is an edge between two nodes in G - c , then these two nodes are said to be independent of each other (line 2). In graph theory, a clique of an arbitrary undirected graph G = (V, E) is a subset of nodes S ⊆ V in which any two nodes of S are connected by an edge in the subgraph G(S); a maximal clique is a clique such that if it is extended by adding any other node, the resulting larger subgraph is not complete. We search for all maximal cliques within G - c , which form a partition π of all FL-PTs V. Any two FL-PTs in a subset Ŝk ∈ π are independent of each other. In line 3, the cliques of G - c can be found by the classic Bron-Kerbosch algorithm [45] .Secondly, for each Ŝh ∈ π, we process the subgraph G b ( Ŝh ) of the benefit graph. All strongly connected components of G b ( Ŝh ) are founded by the classic Tarjan algorithm [44] . The node sets of Definition 2. In the graph Z b , there is a directed edge from vl to vl ′ if and only if there exist two nodes v i ∈ vl and v j ∈ vl ′ such that (v i , v j ) is a directed edge in the benefit graph G b . In the graph Z c , there is an undirected edge between vl and vl ′ if and only if there exist two nodes v i ∈ vl and v j ∈ vl ′ such that (v i , v j ) is an undirected edge in the competing graph G c . For any two coalitions vl and vl ′ of π, vl is said to benefit (resp. benefit from) vl ′ if there is a directed edge (v l , vl ′ ) (resp. (v l ′ , vl )) in the graph Z b ; vl and vl ′ are said to be competitive if there is an undirected edge (v l , vl ′ ) in the graph Z c and independent of each other otherwise.In line 9, Algorithm 2 is called where there is a while-loop. In Algorithm 2, we check whether the loop condition in line 1 is satisfied or not; this loop condition will be called the cycle condition subsequently. As illustrated by v2 and v5 in Figure 2 (a), if "yes", let X = {v yi } θ i=1 denote the nodes of this cycle. The following operations are taken to update the graph Z b and Z c : (i) the nodes of X are merged into a new node vy where all the edges in the graph Z b that point to (resp. point from) the nodes of X change to point to (resp. point from) vy , and (ii) the nodes of X in Z b also exist in the graph Z c ; such nodes in Z c are still merged into a new node vy , and all the edges in the graph Z c whose endpoints are the nodes of X change to become the edges whose endpoints are vy . The above operations will be used as a routine and are given in Algorithm 3.In line 10, Algorithm 4 is called where there are two while-loops. In Algorithm 4, we check whether the loop condition in the outer while loop is satisfied (line 1); this loop condition will be called the Algorithm 2: MergeCycle(π, Z b , Z c , y)1 while there is a node vyi of Z b with |v yi | = 1 such that (i) there is a cycle (v y1 , vy2 , • • • , vy θ ,vy1 ) in the graph Z b that contains vyi and (ii) the nodes vy1 , • • • , vy θ of this cycle are independent of each other do // the cycle condition. 2 (v y , π, Z b , Z c , y) ← Merge(X , π, Z b , Z c , y), presented as Algorithm 3, where X = {v y l } θ l=1 ; 3 Return (π, Z b , Z c , y); Algorithm 3: Merge(X , π, Z b , Z c , y) 1 vy ← vj ∈X vj , y ← y + 1, π ← π -X ,c ; 5 Return (v y , π, Z b , Z c , y); Algorithm 4: MergePath(π, Z b , Z c , y) 1 while there is a node vyi of Z b with |v yi | = 1 such that (i) there is a simple path (v y1 , • • • , vyi , • • • , vy θ ) with vy1 ⩾ 2(v y , π, Z b , Z c , y) ← Merge(X , π, Z b , Z c , y) where X = {v y l } θ l=1 ; 3 (π, Z b , Z c , y) ← MergeCycle(π, Z b , Z c , y), presented as Algorithm 2;4 Return (π, Z b , Z c , y); (v y , π, Z b , Z c , y) ← Merge(X , π, Z b , Z c , y) where X = {v l , vl ′ }; 3 (π, Z b , Z c , y) ← MergeCycle(π, Z b , Z c , y), presented as Algorithm 2; 4 (π, Z b , Z c , y) ← MergePath(π, Z b , Z c , y), presented as Algorithm 4;5 Return (Z b , Z c , y); path condition. If "yes", let X = {v yi } θ i=1 denote the nodes of the simple path. Then, Algorithm 3 is called with the current X as its input. Afterwards, Algorithm 2 is called here to remove that type of cycles defined in line 1 of Algorithm 2. In line 11, Algorithm 5 is called where there are three while-loops. We check whether the loop condition in the outer while loop is satisfied (line 1); this loop condition is called the node condition. As illustrated by v1 and v4 in Figure 2 (a), if "yes", let X = {v l } ∪ {v l ′ }; then, Algorithm 3 is called with X as its input. Afterwards, Algorithms 2 and 4 are called here to guarantee that those types of cycles and simple paths in the cycle and path conditions don't exist in the graph Z b .The time complexity of algorithm 1 depends on the Bron-Kerbosch algorithm in line 3 and is O(3 n/3 ) [45] ; please see Appendix A.1 for a detailed analysis. In line 1 of Algorithm 2 or 4, any two coalitions vy l and vy l ′ are independent (of each other); in line 1 of Algorithm 5, the two coalitions vl and vl ′ are independent. Any two members from such two coalitions respectively will be independent by Definition 2. In each coalition, all its members are independent. Thus, all members of the merged vy are independent of each other. For any coalition vy , the collaboration relationships among its members are established according to the subgraph G b (v y ); thus, we have two conclusions. Firstly, Principle 2 is realized since for the data usage graph G u , paths may exist only between two nodes in a coalition. Secondly, after the merge operation in line 2 of Algorithm 2 or 4, for any vy l with |v y l | = 1, it can benefit from its direct predecessor and benefit its direct successor in the cycle or the simple path, satisfying Eq. ( 1). As a result, we have the following proposition whose detailed proof can be found in Appendix A.2. Proposition 1. Upon completion of Algorithm 1, Principles 1 and 2 are realized.Upon completion of Algorithm 1, the cycle condition, the path condition, and the node condition are not satisfied. Then, we have the following two conclusions: (i) when the cycle condition isn't satisfied, we have for any coalition vyi with |v yi | = 1 that it can be merged with other coalitions without violating Principles 1 and 2 if and only if the path condition is satisfied in the graphs Z b and Z c ; and (ii) When both the cycle condition and the path condition aren't satisfied, we have for any coalition vl with |v l | ⩾ 2 that it can be merged with other coalitions without violating Principles 1 and 2 if and only if the node condition is satisfied in the graphs Z b and Z c . The proofs of the above two conclusions are given in Appendix A. 3 . With these two conclusions, we further have the following proposition.Proposition 2. Upon completion of Algorithm 1, Eq. ( 2) holds.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experimental setup: 
 Datasets & data heterogeneity. We conduct experiments on the CIFAR-10 and CIFAR-100 datasets with different data heterogeneity settings [24] . Also, we use a real-world dataset eICU [37] to illustrate the practicality of FedEgoists. Both the CIFAR-10 and CIFAR-100 datasets contain 60,000 color images for image classification tasks but have different levels of complexity. CIFAR-10 images have 10 classes with 6,000 images per class, while CIFAR-100 is more complex and has 100 classes with only 600 images per class. We simulate the data heterogeneity by two typical approaches: (i) pathological distribution [5, 33, 43, 62] , where each FL-PT is randomly allocated 2 classes of images for CIFAR-10 and 20 classes of images for CIFAR-100, and (ii) Dirichlet distribution [42, 58, 63] , where a distribution vector q c ∈ R n is drawn from the distribution Dir n (β) for each class c and FL-PT v i is allocated a q c,i proportion of data samples of class c; smaller β value results in higher data heterogeneity, and we set β = 0.5. Above, we use standard datasets and randomly divide each dataset into different parts that are used as the local data of different FL-PTs. We also use a real-world dataset that contains data directly from multiple different FL-PTs. Specifically, eICU is a dataset collecting EHRs from many hospitals across the United States admitted to the intensive care unit (ICU). The task is to predict mortality during hospitalization. More details on the way of processing these datasets are given in Appendix B.1. [33] is widely recognized as a vanilla FL algorithm. Multiple representative personalized FL (PFL) methods are used as baselines for comparison [42] . FedProx [26] and SCAFFOLD [22] represent two typical approaches that make the aggregated model at the CS close to the global optima and are two benchmarks in [25] . In other baseline methods, different personalized models are trained for individual FL-PTs. pFedHN [39] and pFedMe [41] represent two approaches based on hypernetworks and meta-learning respectively. FedDisco [59] and pFedGraph [58] are two approaches based on data complementarity, which are proposed recently. FedOra [49] is a method that assesses whether a FL-PT's generalization performance can benefit from knowledge transferred from others and maximizes it. The last baseline is Local where each FL-PT simply takes local ML training without collaboration. These traditional FL approaches cannot address the issues of both self-interest and competition. Thus, we use the operations in lines 1-4 of Algorithm 1 to generate a set of coalitions, denoted as∪ H h=1 SCC h where SCC h = { Ŝh,1 , • • • , Ŝh,y h }. If | Ŝh,1 | ⩾ 2,Ŝh,1 represents a group of independent FL-PTs that can collaborate together following Principles 1 and 2. the above nine FL approaches are applied to each group of such FL-PTs to generate the baseline results.Finally, like [5, 27, 43] , the hypernetwork technique in [34] is used to compute the benefit graph G b and a hypernetwork is constructed by a multilayer perceptron (MLP). The specific way of generating G b is introduced in Appendix B.We conduct experiments on CIFAR-10 and CIFAR-100 with competing graphs that are generated randomly. To simulate competition, the probability of two FL-PTs competing against each other is set to α, thus generating a random competing graph G c [43] , which constrains the collaboration between some FL-PTs; here, the probability that two FL-PTs are independent of each other is 1-α. The value of α determines the intensity of competition among FL-PTs and a larger value reflects a higher level of competing intensity among FL-PTs.Firstly, we verify the effect of the competition intensity on the effectiveness of the proposed solution where we vary the value of α that takes different values in {0.05, 0.1, 0.2, 0.3, 0.4} and conducted the corresponding experiments. Given a specific value of α, we conducted five trials to show the average performance. For the l-th trial, a particular competing graph G c,l is randomly generated with the given α; then, the experiments for the baseline and proposed approaches are run; the performance of the proposed approach is denoted as r α,l,p while the performance of the l-th baseline approach is denoted as r α,l,i , i ∈ {1, 2, • • • , 9}. Given the value of α, we show the average performance of the five trials (i.e., 5 l=1 r α,l,p 5and5 l=1 r α,l,i 5 , i ∈ {1, 2, • • • , 9}).The model performance is measured by the mean test accuracy (MTA), and the experimental results are presented in Tables 1 2 in the form of mean±std; here, PAT and Dir represent the pathological and Dirichlet distributions respectively. It is observed from Tables 1-2 that on average, FedEgoists achieves a significant performance improvement when compared with all the baseline approaches. In Appendix B.4.1, we also illustrate the collaboration relationships among the ten FL-PTs. It is shown that FedEgoists can facilitate the collaboration among FL-PTs and thus achieves a better performance.Secondly, we define a metric to evaluate the worst-case performance across the five trials. In each trial, the nine baseline approaches perform differently. We find an integer l * such that under the l * -th trial, the baseline approaches achieve the best performance, i.e. l * = arg max l∈ [1, 5] max i∈[1,9] r α,l,i where max i∈[1,9] r α,l,i is the best performance of all the nine baseline approaches in the l-th trial. In the trial that is the most advantageous to the baseline approaches, max i∈[1,9] r α,l * ,i -r α,l * ,p is the performance improvement (or difference) of the best baseline approach to the proposed approach, which may be negative if the proposed approach achieves a better performance. Upon identifying l * , we then detail the performance outcomes of both the proposed and baseline models during the l * -th trial. For various values of α, ranging from 0.05 to 0.4, we calculate the performance difference, max i∈[1,9] r α,l * ,i -r α,l * ,p . The results for these computations, under each specified α, PAT Dir CIFAR10 0.011000 -0.002903 0.022900 -0.000624 0.025800 -0.0006030 0.028800 -0.005725 -0.002399 -0.000100 CIFAR100 -0.000999 0.076002 0.011400 -0.000008 -0.000636 -0.0009356 -0.000020 -0.032153 -0.000699 -0.027078 are systematically presented in Table 3 , which shows that in the worst case, the proposed FedEgoists has a performance very close to the best performance of all the baseline approaches.Finally, we also conduct experiments to show the effect of data heterogeneity and the related results can be found in Appendix B.4.2.Following the setting in [5, 43] , there are ten hospitals in total, with {v i } 4 i=0 as large hospitals and {v i } 9 i=5 as small hospitals. Due to the extreme imbalance of data labels, where over 90% are negative labels, we use the AUC scores to evaluate the performance of the trained model. Suppose there are more than one large hospital located in the same city while small hospitals are dispersed in different rural areas with lower population densities; competition mainly occurs among large hospitals. We posit that competitive relationships exist between the following pairs of FL-PTs: (v 1 , v 4 ), (v 2 , v 3 ), and (v 2 , v 4 ). Table 4 presents our experimental results. In Appendix B.5, we also illustrate the collaboration relationships among the 10 FL-PTs. In the naive baseline methods, the formed coalitions are {v 0 , v 3 , v 4 }, {v 1 , v 2 } and {v 5 , v 6 , • • • , v 9 }. In the proposed method, larger coalitions are formed so that their members can benefit from more FL-PTs in the same coalition; the formed coalitions are {v 0 , v 3 , v 4 , • • • , v 9 } and {v 1 , v 2 }. Thus, FedEgoists can facilitate the collaboration among FL-PTs. In Table 4 , it is observed that FedEgoists achieves a better performance on the whole.In addition to CIFAR-10, CIFAR-100 and eICU, we also conduct experiments on the synthetic data like [43] and the related settings and results are presented in Appendix B.6. 
conclusions: 
 In this paper, we consider cross-silos FL where organizations in the business sector that engage in business activities are key sources of FL-PTs. The resulting FL ecosystem has two features: (i) self-interest and (ii) competition among FL-PTs. Correspondingly, two principles are proposed to build a sustainable FL ecosystem: (1) a FL-PT can benefit from the FL ecosystem if and only if it can benefit the FL ecosystem, and (2) a FL-PT will not contribute to its competitors as well as the supporters of its competitors. The Fl ecosystem needs to be tailored to realize these principles. In this paper, we propose an efficient solution that can well realize the two principles above. In the meantime, for any two coalitions without competition that can collaborate together, one coalition cannot increase its utility by collaborating with the other coalition. Extensive experiments over real-world datasets have demonstrated the effectiveness of the proposed solution compared to nine baseline methods, and its ability to establish efficient collaborative networks in cross-silos FL with FL-PTs that engage in business activities.[62] Jianqing Zhang, Yang Hua, Hao Wang, Tao Song, Zhengui Xue, Ruhui Ma, and Haibing Guan.Fedala: Adaptive local aggregation for personalized federated learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11237-11244, 2023.[63] Zhuangdi Zhu, Junyuan Hong, and Jiayu Zhou. Data-free knowledge distillation for heterogeneous federated learning. In International conference on machine learning, pages 12878-12889. PMLR, 2021.A Theoretical Analysis In Algorithm 2, there is a while loop. In line 1, all cycles can be found by the Johnson's algorithm [20] , which is implemented in a Python library and has a time complexity of EQUATIONO ((n + E(Z b ))(c + 1)) + O(cn 2 ) = O (n 2 + E(Z b ))(c + 1). At each iteration of the while loop (line 2), Algorithm 3 is called and the time complexity of Algorithm 3 is O(n 2 ). The number of iterations is upperly bounded by n. Thus, the complexity of Algorithm 2 is no larger thanO n(n 2 + E(Z b ))(c + 1) = O(f 1 (n, E(Z b ), c))whereEQUATIONIn Algorithm 4, there is an outer while loop. In line O((n + E(Z b ))τ ) + O(τ n 2 ) = O (n 2 + E(Z b ))τ = O(f 2 (n, E(Z b ), τ ))whereEQUATIONAt each iteration of the while loop, the operations in lines 2-3 are executed. In line 2, Algorithm 3 is called and the time complexity of Algorithm 3 is O(n 2 ). In line 3, Algorithm 2 is called with a time complexity O (f 1 (n, E(Z b ), c)). The number of iterations is upperly bounded by n. Thus, the complexity of Algorithm 4 isO (n(f 1 (n, E(Z b ), c) + f 2 (n, E(Z b ), τ ))).In Algorithm 5, the time complexity of checking the node condition in line 1 has a time complexity of O(E(Z b )). At each iteration, the operations in lines 2-4 are executed. In lines 2-4, Algorithms 3, 2 and 4 are called sequentially. As analyzed above, these operations have a time complexity no larger than O (n(f1 (n, E(Z b ), c) + f 2 (n, E(Z b ), τ ))).The number of iterations is upperly bounded by n. Thus, the complexity of Algorithm 5 is no larger thanO n 2 (f 1 (n, E(Z b ), c) + f 2 (n, E(Z b ), τ )) .To sum up, since a complete graph of n nodes has a total of n(n -1) that is an upper bound of E(Z b ), where π is the node set of Z b and |π| ⩽ n. Further, we haveEQUATIONEQUATIONThe worst-case time complexity of the operations in lines 9-11 depends on Algorithm 5 and is no larger thanEQUATIONFinally, if c and τ can be upperly bounded by constants, then the time complexity of Algorithm 1 depends on the Bron-Kerbosch algorithm in line 3 and is O(3 n/3 ). We note that this paper studies cross-silo FL in business sectors where there are typically a limited number of FL-PTs (e.g., 2 to 100) [17, 28, 50] . 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We provided the standard deviation of the results in the experimental results.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: In the B.7, We have provided detailed experimental environment.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: The research fully adheres to the NeurIPS Code in all aspects.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: Our work focuses on building a FL ecosystem that can be applied to real-life business scenarios.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: The paper poses no such risks.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: When comparing with the baseline algorithm, its code was used and this article was cited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A graph G is said to be strongly connected if it contains a directed path from v i to v j and a directed path from v j to v i for every pair of nodes v i and v j of G. Specially, a graph that contains only a single node is said to be strongly connected trivially. Lemma 1. Principle 1 is realized when Eq. ( 1) is satisfied.Proof. Only the FL-PTs in the same coalition may collaborate. For any FL-PT v i ∈ V, there exists a coalition S k ∈ π such that v i ∈ S k . When |S k | = 1, the only FL-PT v i neither receives nor provides benefits to any other FL-PT, thereby satisfying Principle 1 trivially. When |S k | ⩾ 2, v i engages in a reciprocal exchange of benefits with the other members of S k -{v i }, which is guaranteed by Eq. (1). Thus, v i both contributes to and benefits from other members and Principle 1 is satisfied. ■ Lemma 2. Upon completion of lines 1-6 of Algorithm 1, we have for any vl ∈ π with |v l | ⩾ 2 that (i) all FL-PTs of the coalition vl are independent of each other and (ii) Eq. ( 1) holds for any FL-PT v i ∈ vl . . The merge operation occurs (i.e., Algorithm 3 is executed) if the cycle condition, the path condition and the node condition are satisfied in Algorithms 2, 4 and 5 respectively. Upon each completion of Algorithm 3, π is updated: some coalitions will be merged into a new coalition vy that is further added to π, and then be removed from π. Like Lemma 2, we still have the following conclusion.Lemma 3. At the beginning of each execution of Algorithm 3, suppose we have for any vl ∈ π with |v l | ⩾ 2 that (i) all FL-PTs of the coalition vl are independent of each other and (ii) Eq. ( 1) holds for any FL-PT v i ∈ vl . Then, upon each completion of Algorithm 3, we still have for any vl ∈ π with |v l | ⩾ 2 that (i) all FL-PTs of the coalition vl are independent of each other and (ii) Eq. ( 1) holds for any FL-PT v i ∈ vl .Proof. Algorithm 3 is called if the cycle condition, the path condition and the node condition are satisfied in Algorithms 2, 4 and 5 respectively.In line 1 of Algorithms 2 or 4, any two coalitions vy l and vy l ′ are independent of each other; by Definition 2, any two members from vy l and vy l ′ respectively are independent of each other. In the meantime, for any vy l ∈ π, any two members of vy l are independent of each other. Thus, upon completion of the merge operation in line 2 of Algorithms 2 or 4, vy1 , vy2 , • • • , vy θ are merged into a new coalition vy , and all FL-PTs of vy are independent of each other. This completes the proof of the first point of Lemma 3 when it comes to Algorithms 2 and 4.Similarly, in line 1 of Algorithm 5, any two members from vl and vl ′ respectively are independent of each other. In the meantime, for any vy l ∈ π, any two members of vy l are independent of each other. Thus, upon completion of the merge operation in line 2 of Algorithm 5, vl and vl ′ are merged into a new coalition vy , and all FL-PTs of vy are independent of each other. This completes the proof of the first point of Lemma 3 when it comes to Algorithm 5.The collaboration relationships among the members of vy are established exactly according to the subgraph G b (v y ). In line 1 of Algorithms 2 or 4, for any l ∈ [1, θ], if |v y l | ⩾ 2, the members of vy l already satisfy Eq. ( 1). Let us consider any two nodes vl and vl ′ in the graph Z b . By Definition 2, if there is an edge from vl to vl ′ in the graph Z b , then there is a node of vl ′ that benefits from a node of vl in the benefit graph G b . If |v y l | = 1, the single node in vy l will benefit its direct successor in the cycle (resp. the simple path) and benefit from its direct predecessor in this cycle (resp. the simple path); thus the node of vy l also satisfies Eq. ( 1). This completes the proof of the second point of Lemma 3 when it comes to Algorithms 2 and 4. In line 1 of Algorithm 5, since |v l | ⩾ 2 and |v l ′ | ⩾ 2, the members of vl and vl ′ already satisfy Eq. ( 1). This completes the proof of the second point of Lemma 3 when it comes to Algorithm 5. ■ Finally, by Lemma 2, we have the following conclusion holds upon completion of lines 1-6 of Algorithm 1: for any vl ∈ π with |v l | ⩾ 2, (i) all FL-PTs of the coalition vl are independent of each other and (ii) Eq. ( 1) holds for any FL-PT v i ∈ vl . Afterwards, Algorithm 3 may be executed multiple times by the call by Algorithms 2, 4 and 5; upon each completion of Algorithm 3, π is updated such that some of its coalitions are removed from π and merged into a larger one that is added to π. By Lemma 3, upon completion of Algorithm 1, the conclusion above still holds. Since Eq. ( 1) holds, Principle 1 is realized by Lemma 1. For any vl ∈ π, the collaboration relationships among the members of a coalition vl are established exactly according to the subgraph G b (v l ). Each FL-PT v i of a coalition only collaborate with other FL-PTs within the same coalition, The data usage graph G u in fact consists of multiple separated subgraphsG b (v l ), i.e., G u = {G b (v l ) | vl ∈ π}.In the graph G u , paths may exist only between two FL-PTs of the same coalition, and any two FL-PTs in the same coalition are independent of each other. Thus, Principle 2 is realized.After finishing the operations in line 9 of Algorithm 1, the loop condition in line 1 of Algorithm 2 (i.e., the cycle condition) is not satisfied. The execution in line 10 of Algorithm 1 starts. Before executing the first iteration of the while loop of Algorithm 4, if the loop condition (i.e., the path condition) is not satisfied, then Algorithm 4 ends with both the cycle condition and the path condition violated. If Algorithm 4 ends after executing several iterations, then the cycle condition (line 3) and the path condition (line 1) are both violated.Subject to Principles 1 and 2, no coalitions of π (i.e., no subset π ′ of π) can collaborate together and be merged into a larger coalition with a higher utility; in other words, after merging these coalitions into a larger one, no FL-PTs in these coalitions π ′ increase their utilities under the larger coalition.Formally, like what we define in Section 3.3, let Proof. Firstly, we prove the "only if" direction by contradiction. For a coalition vyi that contains only one FL-PT/node, it can collaborate with other coalitions without violating Principle 1 only if there are two other coalitions vyi-1 and vyi+1 that can both benefit and benefit from it; by Definition 2, in the case that vyi-1 and vyi+1 denote the same coalition, there is a cycle in the graph Z b between vyi and vyi-1 where vyi-1 = vyi+1 . Suppose there are some coalitions that can collaborate with vyi without violating Principles 1 and 2 (i.e., there exists a non-empty π ′ ∈ Π such that vyi ∈ π ′ ); correspondingly, in the graph Z b , there will be a path of the formΠ =    π ′ ⊆ π | vy i ∈π ′ u(v yi ) < u   vy i ∈π ′ vyi   ,(v y1 , • • • , vyi-1 , vyi , vyi+1 , • • • , vy θ )in which a coalition will benefit its successor; the nodes of this path are also independent of each other according to the graph Z c in order to be able to collaborate together without violating Principle 2. Then, this path is simple and not a cycle (i.e., vy1 ̸ = vy θ ); otherwise, it is a cycle that has two nodes that compete against each other. In this path, all intermediate nodes can both benefit and benefit from other nodes. Each of vy1 and vy θ also represents a coalition and should contain at least two nodes of V; then, each node in vy1 and vy θ already satisfies Eq. ( 1 ′ = {v y1 , • • • , vyi , • • • , vy θ }. For any l ∈ [2, θ],there exists at least one FL-PT v i of vy l that can benefit from some FL-PT in vy l , which is denoted as v j by Definition 2. Before merging the coalitions of π ′ , the FL-PT v i of a coalition vy l ∈ π ′ only collaborates with the other FL-PTs with the same coalition vy l and the collaboration relationship is established according to G b (v y l ). Let vy = vy i ∈π ′ vyi . After merging the coalitions of π ′ into a larger coalition vy , FL-PT v i can additionally benefit from FL-PT v j , i.e., u i (v y l ) < u i (v y ); in the subgraph G b (v y ), v i can benefit from all FL-PTs of vy such that they point to v i by directed edges. This completes the proof of the "if" direction. ■After finishing the operations in line 10 of Algorithm 1, the cycle condition and the path condition cannot be satisfied. Then, the execution in line 11 of Algorithm 1 starts. Before executing the first iteration of the while loop of Algorithm 5, if the loop condition (i.e., the node condition) is not satisfied, then Algorithm 5 ends with the cycle, path and node conditions violated. If Algorithm 5 ends after executing several iterations, then the cycle and path conditions (lines 3-4) and the node condition (line 1) are violated.Subject to Principles 1 and 2, no coalitions of π (i.e., no subset π ′ of π) can collaborate together and be merged into a larger coalition with a higher utility; in other words, after merging these coalitions into a larger one, no FL-PTs in these coalitions π ′ increase their utilities under the larger coalition.Formally, like what we define in Section 3.3, letΠ = π ′ ⊆ π | vl ∈π ′ u(v l ) < u vl ∈π ′vl , Principles 1 and 2 are satisfied byvl ∈π ′ vl .Given a graph Z b where the cycle, path and node conditions are not satisfied, we have the following conclusion. Lemma 5. Suppose the graph Z b is a graph without any such path that (i) the nodes of this path are all independent of each other and (ii) this path is either a cycle in which there exists a node vy with |v y | = 1, or a simple path of the form(v y1 , • • • , vyi , • • • , vy θ ) with |v yi | = 1, |v y1 | ⩾ 2 and |v y θ | ⩾ 2.Let us consider an arbitrary node vl of Z b with |v l | ⩾ 2; each node vl also represents a coalition; then, we have• The coalition vl can benefit or benefit from other coalitions by being merged with them without violating Principles 1 and 2 (i.e., there exists a non-empty π ′ ∈ Π such that vl ∈ π ′ ) if and only if there exists another node vl ′ with |v l ′ | ⩾ 2 that connects vl by an edge in the graph Z b and the two nodes vl and vl ′ are independent of each other.Proof. Firstly, we prove the "if" direction. If there exists another node vl ′ with |v l ′ | ⩾ 2 that connects vl by an edge in the graph Z b and the two nodes vl and vl ′ are independent of each other, then the coalitions that these two nodes in Z b represent can be merged into a new subset vy of nodes of V; doing so doesn't violate Principle 2 by Definition 2. Each FL-PT in vl and vl ′ already satisfies Eq. ( 1) since any two FL-PTs of G b (v l ) or G b (v l ′ ) are connected. Thus, any node in the merged set vy still satisfies Principle 1. Finally we have π ′ = {v l , vl ′ }. This completes the proof of the "if" direction.Secondly, we prove the "only if" direction. For a coalition vyi that contains only one FL-PT/node, it can collaborate with other coalitions without violating Principle 1 only if there are two other coalitions vyi-1 and vyi+1 that can both benefit and benefit from it. Suppose there are some coalitions that collaborate with vyi without violating Principles 1 and 2 (i.e., there exists a non-empty π ′ ∈ Π such that vl ∈ π ′ ); correspondingly, in the graph Z b , there will be a path of the form(v y1 , vy2 , • • • , vyi ), (v yi , vyi+1 , • • • , vy θ ), or (v y1 , • • • , vyi-1 , vyi , vyi+1 , • • • , vy θ )in which a coalition will benefit its successor; the nodes of this path are also independent of each other according to the graph Z c in order to be able to collaborate together without violating Principle 2. Then, by the assumptions in Lemma 5, this path is a simple path in which there doesn't a node such that (i) the coalition that it represents contains only one node of V and (ii) there don't exist one of its upstream nodes and one of its downstream nodes whose corresponding coalitions both contain at least two nodes V. This means that the coalitions that the nodes of this path represent all contain at least nodes. This completes the proof of the "only if" direction. ■As shown above, upon completion of Algorithm 1, the cycle, path and node conditions are not satisfied in the graph Z b . Firstly, by Lemma 4, for arbitrary multiple coalitions in which there is at least one coalition that contains only one FL-PT, they cannot be merged into a larger coalition without violating Principles 1 and 2. Secondly, by Lemma 5, for arbitrary multiple coalitions each of which has at least two FL-PTs, they cannot be merged into a larger coalition without violating Principles 1 and 2. Thus, Proposition 2 holds.B.1 Data Processing CIFAR-10 & CIFAR-100. In both CIFAR-10 and CIFAR-100, there are a total of 50,000 images for training and 10,000 images for testing. In CIFAR-10, each class has 5000 training images and 1000 testing images. In CIFAR-100, each class has 500 training images and 100 testing images. In both the CIFAR-10 and CIFAR-100 experiments, we split the data into 10 FL-PTs, and the training data for each FL-PT is divided into a training set (90%) and a validation set (10%). The label distribution of CIFAR-10 is illustrated in Figure 3 . In Figure 3 , the label distribution across different FL-PTs/FL-PTs indicates that there is a significant variance in the total number of samples per FL-PT, and the data is imbalanced. Similarly, the label distribution of CIFAR-100 is illustrated in Figure 4 . eICU. We process the eICU dataset following the data pre-processing steps in [19, 31] , finally generating JSON files that include hospital identifiers. We divided the training, validation, and testing sets in a ratio of 7:1.5:1.5, with each hospital being treated as an independent FL-PT. We chose a binary classification task using the eICU database, aiming to predict patient mortality within 48 hours of ICU admission(mort_48h) based on the initial data provided.The local model information is mainly leveraged in the calculation of G b . Specifically, there are n FL-PTs. Each FL-PT v i has a risk/loss function ℓ i : R n → R + . Given a learned hypothesis h ∈ H, let the loss vector ℓ(h) = [ℓ 1 , . . . , ℓ n ] represent the utility loss of the n FL-PTs under the hypothesis h. The hypothesis h is considered a Pareto solution if there is no other hypothesis h ′ that dominates h, i.e.,EQUATIONLet r = (r 1 , . . . , r n ) ∈ R n denote a preference vector which denotes the weight of the objective local model loss that is normalized with n k=1 r k = 1 and r k ≥ 0, ∀k ∈ {1, . . . , n}. The hypernetwork HN takes r as input and outputs a Pareto solution h, i.e.,EQUATIONwhere ϕ denotes the parameters of the hypernetwork [23] . For each FL-PT v i , linear scalarization can be used. Like [5, 30] , an optimal preference vector r * i = r * i,1 , r * i,2 , . . . , r * i,n is determined to generate the hypothesis h * i that minimizes the loss with the data Di . This is expressed as For each FL-PT i, the value of r * i,j is used as an estimate to the weight of v j to v i [6, 27] .EQUATION{r * i } n i=1defines a directed weighted graph, i.e., the benefit graph G b .The network architectures used in different datasets are introduced below [5, 15, 16, 31, 35, 43] .When it comes to a specific dataset, all approaches have the same network structure for each FL-PT to execute the learning tasks.Hypernetwork. We construct a hypernetwork that employs a 2-layer hidden MLP to generate the parameters of the target network.CIFAR-10. The target network passes through two convolutional layers, each followed by a 2 × 2 max pooling layer. Afterward, the data is flattened and further processed through two fully connected layers, each followed by a LeakyReLU activation function, finally yielding the prediction results.CIFAR-100. Initially, the input data passes through two convolutional layers, activated by ReLU and each followed by a 2x2 max pooling layer to reduce spatial dimensions and highlight important features. After these convolutional and pooling layers, the data is flattened to prepare for dense processing. Subsequently, it is fed into two fully connected layers, each incorporating a ReLU activation function to add non-linearity and improve learning capabilities. The output is finally produced by a linear layer.We employ a single-layer MLP hypernetwork, while the target network utilizes a Transformer classifier [47] with Layer Normalization [2] . ). This is illustrated in Figure 5 (a). The final collaboration relationships among FL-PTs are illustrated in Figure 5 : Data heterogeneity is typically simulated by a pathological or Dirichlet distribution. For example, the pathological distribution is used in [5, 43] . In this paper, we conducted experiments on pathological and Dirichlet distributions respectively. In addition, We also verify the effect of the data heterogeneity level on the effectiveness of the proposed solution. Specifically, let m denote the number of classes. In Dirichlet distribution, there is a distribution vector q c ∈ R m is drawn from the Dirichlet distribution Dir m (β) for each class c and FL-PT v i is allocated a q c,i proportion of data samples of class c; smaller β value results in higher data heterogeneity. We vary the value of β that takes different values in {0.01, 0.1, 0.5} and conducted the corresponding experiments. The related experimental results are presented in Table 5 . It is observed that, in some cases of high data heterogeneity, FedEgoists achieves a performance close to the Local approach; overall, FedEgoists performs the best when it is compared to the baseline approaches. We present our experimental results with synthetic data across different scenarios with fixed competing graphs like [43] . There are n = 8 FL-PTs. The synthetic data are generated by x ∼ U[-1.0, 1.0]. The noise ϵ ∼ N [0.0, 0.5 2 ] is added to each label. Given the FL-PT v i , the grand truth weights u i,l = v + r i,l are sampled as v ∼ U[0.0, 1.0] and r i,l ∼ N [0.0, ρ 2 ] where l = 1, 2, 3; ρ 2 measures the data distribution discrepancy among FL-PTs. We set ρ = 0.01, which means that the generated data are weakly non-iid in terms of sample features and labels. We construct the hypernetwork using a 1-layer MLP for training the Pareto Front of all objectives. The target network consists of a two-layer linear network. After the first linear layer, a Leaky ReLU activation function is applied. Justification: As stated in the last part of the abstract and introduction, we clearly summarized the contribution and scope of this paper.0 v 1 v 2 v 3 v 4 v 5 v 6 v 0 v 1 v 2 v 3 v 4 v 5 v 6 v 0 v 1 v 5 v 0 v 1 v 5 vGuidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: In this paper, the authors focus on the selection of federated learning participants and study the problem of free riders and conflicts of interest among competitors. Now, the paper has received four reviews: one Accept, two Weak Accept, and one Borderline Reject.

From the positive points of the reviews, the reviewers have generally agreed that (1) the paper has studied an important problem in federated learning, (2) there are some interesting results, and (3) the paper is easy to follow. However, at the same time, the reviewers have pointed out several weaknesses on the evaluation, complexity, novelty, motivation, etc.

During the rebuttal, the authors have provided feedback to each reviewer on these weaknesses, and most of the reviewers remain positive after reading the rebuttal. Moreover, the rebuttal has clarified quite a few concerns raised by the reviewers. Since the overall scores are positive and above the acceptance threshold, we recommend accepting this paper for NeurIPS 2024.

Reviews: review: paper_summary: paper_summary: This paper focuses on client selection in cross-silo federated learning. The authors propose FedEgoists. In particular, FedEgoist participate clients into different clusters to avoid free riders and conflict of interests. Theoretical analysis is provided to validate the theoretical soundness of the proposed FedEgoist.
Experiments validate the effectiveness of proposed FedEgoist.
strength_weakness: strength_weakness: * The evaluation setting on CIFAR-10 and CIFAR-100 cannot simulate conflict of interests in the real world. Further consideration needed.
* The meaning of propositions 1 & 2 is unclear. Further explanation is expected.
questions: questions: Please refer to weaknesses.
limitations: limitations: There is no potential negative societal impact of their work.
paper_summary: paper_summary: The business sector is a main domain where cross-silo federated learning (FL) has many promising applications in various scenarios. The authors simultaneously consider the self-interest and competition features in the business sector. They develop a novel framework to both address the resulting free-riding problem and avoid the conflict of interest between any two competing FL participants (FL-PTs), which results in an interesting optimization problem. Finally, the authors find an optimal solution where one coalition cannot increase the utility of any of its members by collaborating with other coalitions. Extensive experiments are conducted to show the effectiveness of the proposed solution, and its ability to establish efficient collaborative networks in cross-silo FL with FL-PTs that engage in business activities.
strength_weakness: strength_weakness: W1: The proposed algorithm is not polynomial-time solvable. However, this seems reasonable since there are typically a limited number of FL-PTs (e.g., 2 to 100) for cross-silo FL in business sectors. The authors propose a novel application of the classic algorithms in graph theory to solve their problem optimally. The algorithm complexity depends on these classic graph algorithms, which work well in reality.
questions: questions: Q1: Can the proposed technical framework be applied to other types of tasks or datasets, e.g., NLP? In the experimental evaluation part, the authors have already shown its application to some interesting tasks commonly seen in literature.
Q2: To keep consistency, it is better that “selfish” in Figure 1 is changed to “self-interest”.
Q3: In this paper, all FL-PTs are partitioned into mutually disjoint coalitions/groups. It seems that this is similar to clustered federated learning where FL-PTs are partitioned into mutually disjoint clusters (i.e., groups). The reviewer understands that the authors consider a problem that is clearly different from the typical clustered federated learning. 
Q4: In your experimental results, how many trials have you conducted to obtain the standard deviation?
limitations: limitations: See weaknesses and Questions.
paper_summary: paper_summary: This paper studies an interesting topic, which is about cooperation and competition in federated learning. It can be used to describe or simulate the real-world scenarios. The authors propose to use graph-related techniques to formulate the relation between the local clients. They test the algorithm on the benchmark datasets and the medical dataset.
strength_weakness: strength_weakness: 1, The novelty of each module, especially the graph part, may need to be clarified clearly. Also, the two principles are commonly used, if the reviewer understand correctly, an optimal formation is proposed based on these two, is that correct?

2, Most designed part is at the server side. The reviewer may not be able to observe the strong connection with federated learning. The setting itself may tell that each client is a participant in this framework. However, the introduction (motivation) and the method could be a little bit fragmented.

3, Ablation study and hyperparameter study may help verify the algorithm more deeply and soundly.

4, A little bit concerned about the baseline selection. Some contribution-related work in FL like [1], if comparable, should be considered or at least discussed.

[1] Contribution-Aware Federated Learning for Smart Healthcare
questions: questions: 1, Could you please address the concerns in the weakness part?

2, Could you clarify how the local model information is leveraged and updated in the graph part more clearly?

3, How to validate or verify the meaning of the graph construction part? Do we need to have the groundtruth of the relation of all the clients?
limitations: limitations: 1, The connection of the proposed techniques should be closely associated with the setting of FL, which focuses on the iterative updates between the server and the clients.

2, The original novelty of contribution should be emphasized a little bit more.

3, More comprehensive experiments may help demonstrate the effectiveness and understand the generalization of the algorithm better.
paper_summary: paper_summary: This paper introduces a strategy named FedEgoists designed to enhance collaboration in cross-silo federated learning (FL) scenarios, particularly in business sectors where participants (FL-PTs) are often competitive and self-interested. The proposed FedEgoists strategy presents a sophisticated and theoretically sound approach to managing collaborations in cross-silo federated learning, and its strengths lie in its ability to handle self-interest and competition effectively.
strength_weakness: strength_weakness: The paper lacks rigorness as pointed out in the Questions part.
questions: questions: (1) FedEgoists requires a central server to coordinate and enforce the coalition formation, which introduces a single point of control and potential failure. This centralization might conflict with the decentralized nature of federated learning and violate the fundamental motivation of FL.
(2) FedEgoists assumes that FL-PTs will accurately report their competitive relationships to the central server (CS). However, this can easily lead to privacy leaks and potential attacks.
(3) While the FedEgoists strategy focuses on collaboration benefits and competition avoidance, it does not explicitly address data complementarity.
(4) Since the FedEgoists strategy relies on coalition formation based on reported benefits and competition, there is a risk of strategic manipulation by participants.
(5) The experiments are not sufficient, and there are too few datasets and baselines.
limitations: limitations: It seems limitations are not clearly or explictly elaborated.

Comments: title: Thank you!
Discussion Period & Thank you!
Thank you!
Thank you!
comment: Dear Reviewer BcXD, we thank you sincerely for your time to check the rebuttal and your recognization of our work.
Thanks for the detailed responses. I'm generally fine with the results.  Thus, I will keep my score.
Dear Reviewer jdA3, we thank you for your time and detailed comments. Since the discussion period will end soon, we will appreciate if you could check the rebuttal and let us know whether our response has addressed your concerns. Thank you sincerely.
Dear Reviewer 2xqy, we thank you sincerely for your time to check our rebuttal and your positive feedbacks.
Dear Reviewer gR3Y, we thank you sincerely for your time to check our rebuttal. We are glad that our response has addressed your concerns.
Thanks for the author's comments. I think the comments have addressed my concerns and answered the questions. I will increase the score.
Not fully clearing my concern but I think I can increase my rating
**Limitations 3**

Due to space limitations, please refer to the global rebuttal.
**Questions 2 \& 3** 

This paper has two graphs (i.e., the benefit graph $\mathcal{G_b}$, and the competing graph $\mathcal{G_c}$) that are assumed to be known to a FL manager; $\mathcal{G_b}$ is obtained by collaborative model training, while the information on $\mathcal{G_c}$ is reported by clients to the FL manager (or the central server). Below, we detail the ways of obtaining them and how the local model information is leveraged in the graph part.

Firstly, data heterogeneity/complementarity between clients is characterized by a benefit graph $\mathcal{G_b}$. Like [5,30], this paper uses the hypernetwork technique in [23] to evaluate the data complementarity, thus obtaining the benefit graph. In the graph part, the local model information is mainly leveraged in the calculation of $\mathcal{G_b}$. We also refer readers to Section 4.2 of Ref. [5] for the technical details on the way of obtaining $\mathcal{G_b}$. 

Specifically, there are $n$ clients. Each client $v_{i}$ has a risk/loss function $\ell_{i}$: $\mathbb{R^n}\rightarrow \mathbb{R_+}$. Given a learned hypothesis $h\in$ $\mathcal{H}$, let the loss vector $\mathbf{\ell}(h)=[\ell_{1}, \dots, \ell_{n}]$ represent the utility loss of the $n$ clients under the hypothesis $h$. The hypothesis $h$ is considered a Pareto solution if there is no other hypothesis $h^{\prime}$ that dominates $h$, i.e., $\nexists h^{\prime}\in \mathcal{H}, s.\,t.\, \forall i: \ell_{i}(h^{\prime})\leqslant \ell_{i}(h) \text{ and } \exists j: \ell_{j}(h^{\prime}) < \ell_{j}(h). $
Let $r=(r_{1}, \dots,r_{n})$ $\in \mathbb{R^n}$ denote a preference vector which denotes the weight of the objective local model loss that is normalized with $\sum_{k=1}^{n}{r_{k}}=1$ and $r_{k}\geq 0, \forall k\in \{1, \dots, n\}$. The hypernetwork $HN$ takes $r$ as input and outputs a Pareto solution $h$, i.e., $h\gets HN(\phi, r)$, where $\phi$ denotes the parameters of the hypernetwork [23]. For each client $v_{i}$, linear scalarization can be used. Like [5,30], an optimal preference vector $r_{i}^{\ast}=\left(r_{i,1}^{\ast}, r_{i,2}^{\ast}, \dots, r_{i,n}^{\ast}\right)$ is determined to generate the hypothesis $h_{i}^{\ast}$ that minimizes the loss with the data $\hat{\mathcal{D_i}}$. This is expressed as 
$h_{i}^{\ast}=HN(\phi, r_{i}^{\ast}), \text{ where } r_i^* = argmin_r \hat{\mathcal{L_i}}(HN(\phi, r)).$
For each client $v_{i}$, the value of $r_{i,j}^{\ast}$ is used as an estimate to the weight of $v_{j}$ to $v_{i}$ [5,30]. $r_{1}^{\ast}, r_{2}^{\ast}, \dots, r_{n}^{\ast}$ define a directed weighted graph, i.e., the benefit graph $\mathcal{G_b}$. 

Secondly, following the standard technical practices in literature, the central server is assumed to be trustable, as explained above. In real worlds, the central server may represent an impartial and authoritative third-party (e.g., the industry association) [5]. Then, FL-PTs can report their competitive relationships to the third-party in person and confidentiality agreements can be signed between the third-party and FL-PTs. 

The above information will be clarified in the final version of our manuscript.
**Notes on other added experiments.** In this rebuttal, besides the experiments conducted for the above comments in W1, more experiments are also conducted (a) to verify the robustness of the proposed solution in terms of the level of data heterogeneity, and (b) on a new dataset and with a new baseline approach.

**Experiments (a)**

Specifically, this paper is about the application of the current FL techniques to the business sectors, and the experiments aim to verify the effectiveness of the proposed theoretical framework in the scenario under study. The scenario features self-interest for every client and competition between a part of clients. The collaboration relationships between clients are built according to the competing and benefit graphs; the benefit graph depends on the data heterogeneity/complementarity of clients. Thus, the variable factors in the scenario/experimental environment include the intensity of competition between clients and the non-IID setting. In the experiments, we vary the related parameters in these two factors to better verify the robustness of the proposed algorithm. Above, the experiments for the intensity of competition have been introduced.  

Data heterogeneity is typically simulated by a pathological or Dirichlet distribution. For example, the pathological distribution is used in [5,30]. In this paper, we conducted experiments on pathological and Dirichlet distributions, respectively. Also, in this rebuttal, we better verify the effect of data heterogeneity on the effectiveness of the proposed solution. Specifically, let $m$ denote the number of classes. In Dirichlet distribution, there is a distribution vector $q_{c}\in \mathbb{R}^{m}$ is drawn from the Dirichlet distribution $Dir_{m}(\beta)$ for each class $c$ and client $v_{i}$ is allocated a $q_{c,i}$ proportion of data samples of class $c$; smaller $\beta$ value results in higher data heterogeneity. We vary the value of $\beta$ that takes different values in {0.01,0.1,0.5} and conducted the corresponding experiments. The related experimental results are presented in Table 4.

**Experiments (b)**

We additionally conducted experiments on the synthetic data used in [30] where the experimental setting is also the same as the setting in [30]. The related experimental results are presented in Tables 5-6, which can be found in the PDF document. We add one more approach in the paper below, called  FEDORA, as the baseline:

Jun Wu, Wenxuan Bao, Elizabeth Ainsworth, and Jingrui He. "Personalized federated learning with parameter propagation." In ACM KDD, 2023.

FEDORA is only for image classification tasks; thus, we only conducted experiments on CIFAR-10 and CIFAR-100. The related experimental results are presented in Tables 1,2,4.
**Weaknesses 4** 

The two papers have different focuses. There is a central server that coordinates the FL training process. In [1], a global ML model is built for all clients and the paper aims to evaluate the individual contribution of each client/FL-PT to this global model. In our paper, a personalized ML model is produced for each individual client $v_{i}$ and the central server determines which clients can contribute to this client $v_{i}$ in the FL training process. While determining such contribution relationships between clients, we aim to avoid free-riding and conflicts of interests between clients.