Title: Resolving the Tug-of-War: A Separation of Communication and Learning in Federated Learning
Venue: NeurIPS 2023
Accepted: yes
Keywords: Federated Learning
Abstract: Federated learning (FL) is a promising privacy-preserving machine learning paradigm over distributed data. In this paradigm, each client trains the parameter of a model locally and the server aggregates the parameter from clients periodically. Therefore, we perform the learning and communication over the same set of parameters. However, we find that learning and communication have fundamentally divergent requirements for parameter selection, akin to two opposite teams in a tug-of-war game. To mitigate this discrepancy, we introduce FedSep, a novel two-layer federated learning framework. FedSep consists of separated communication and learning layers for each client and the two layers are connected through decode/encode operations. In particular, the decoding operation is formulated as a minimization problem. We view FedSep as a federated bilevel optimization problem and propose an efficient algorithm to solve it. Theoretically, we demonstrate that its convergence matches that of the standard FL algorithms. The separation of communication and learning in FedSep offers innovative solutions to various challenging problems in FL, such as Communication-Efficient FL and Heterogeneous-Model FL. Empirical validation shows the superior performance of FedSep over various baselines in these tasks.

Resolving the Tug-of-War: A Separation of Communication and Learning in Federated Learning

Abstract

Federated learning (FL) is a promising privacy-preserving machine learning paradigm over distributed data. In this paradigm, each client trains the parameter of a model locally and the server aggregates the parameter from clients periodically. Therefore, we perform the learning and communication over the same set of parameters. However, we find that learning and communication have fundamentally divergent requirements for parameter selection, akin to two opposite teams in a tug-of-war game. To mitigate this discrepancy, we introduce FedSep, a novel two-layer federated learning framework. FedSep consists of separated communication and learning layers for each client and the two layers are connected through decode/encode operations. In particular, the decoding operation is formulated as a minimization problem. We view FedSep as a federated bilevel optimization problem and propose an efficient algorithm to solve it. Theoretically, we demonstrate that its convergence matches that of the standard FL algorithms. The separation of communication and learning in FedSep offers innovative solutions to various challenging problems in FL, such as Communication-Efficient FL and Heterogeneous-Model FL. Empirical validation shows the superior performance of FedSep over various baselines in these tasks.

1 Introduction

In Federated Learning (FL) [37], a set of clients jointly solve a machine learning task under the coordination of a central server. The process of FL involves two category of operations: Learning and Communication. For the learning operation, each client optimizes their local objectives, and for the communication operation, clients exchange local parameters to facilitate knowledge sharing. In the existing FL pipeline, both Learning and Communication operations hinge on the same set of parameters. However, these two operations have fundamentally divergent requirements, akin to two teams engaged in a tug-of-war, each pulling in opposite directions. In fact, on the communication side, it is imperative that the parameters of all clients reside in a uniform space. Moreover, to mitigate the high communication costs, a major bottleneck in FL, it is beneficial to maintain these parameters within a low-dimensional space. In contrast, on the learning side, given the heterogeneity of devices, including variations in hardware and data distribution, it is advantageous to allow the parameter space to differ across clients. Furthermore, to accommodate the implementation of state-of-the-art large-scale machine learning models (such as Transformers [47]), a high-dimensional parameter space is desirable. In summary, a huge discrepancy of requirements to parameter selection exists between the communication and the learning in FL.

To mitigate this discrepancy, we opt to 'break the rope' in this tug-of-war scenario. More precisely, we propose a two-layer structure on clients: a communication layer and a learning layer. As shown in Figure 1, we denote this framework as FedSep : The communication layer connects with the server and uploads/downloads parameter, while the learning layer performs local objective optimization. Then the communication layer and the learning layer are connected through encode/decode operations. The decode operation maps the parameter of the communication layer to the parameter of the learning layer and the encode operation performs the mapping in the opposite direction. More specifically, we formulate the decode operation as solving a minimization problem and does not assume an explicit relation between the parameter of communication layer and the parameter of the learning layer. In summary, we aim to use two distinct sets of parameters in FedSep to resolve the 'competition' between the communication and learning operations, meanwhile, the two set of parameters are close connected through decode/encode operations. Mathematically, FedSep can be formulated as solving a federated bilevel problem. The upper level problem corresponds to the learning problems on clients, while the lower level problem is the minimization problem in the decode operation. We propose an efficient algorithm to solve this bilevel problem, which is composed of three consecutive steps: Decode the communication parameter, Learning on local problems and Encode the learning parameter to the communication parameter. The convergence of the algorithm is guaranteed.

To demonstrate the flexibility of FedSep in incorporating the contradictory requirements of the communication and learning, we study two real-world FL tasks. In the first task, we study the communication-efficient FL task. In this task, we set the dimension of the communication parameter to be much smaller than the learning parameter, furthermore, we decode the communication parameter through a LASSO problem. In the second task, we study the modelheterogeneous FL task. In this task, the learning parameters on different clients have different dimension. We set the communication parameter to be the parameter of the server model, and the learning parameter be a subset of the communication parameter (server model). In particular, the decode operation is to select the subset adaptively based on the client's local data. We empirically verify the superior performance of our methods compared to other baselines. Finally, we summarize the main contributions of our work as follows:

Server

Upload

Download

Communication

Layer

Decode

Encode

Client1

Clientm

ClientM

We propose a novel two-layer Federated Learning framework, i.e. FedSep , where the communication layer and the learning layer are separated on clients;

We let the decode operation from the communication layer to the learning layer be solving a minimization problem, and therefore, the FedSep framework has a bilevel optimization formulation. We propose an efficient algorithm to solve the bilevel problem and show that its convergence rate matches that of the standard Federated Learning algorithms;

We apply the FedSep framework to solve two important FL tasks: Communication-Efficient FL and Model-Heterogeneous FL. Numerical experiments show the superior performance of the methods based on FedSep.

Notations. ∇ denotes the full gradient, ∇ x is the partial derivative for variable x, and higher-order derivatives follow similar rules. || · || 2 is ℓ 2 -norm for vectors and the spectral norm for matrices. AB denotes the multiplication of the matrix between the matrix A and B . [ K ] represents the sequence of integers from 1 to K .

2 Related Works

Federated Learning. FL is a novel paradigm for performing machine learning tasks over distributed data, but it also poses challenges such as heterogeneity [24], privacy [38] and communication

bottleneck [48]. In particular, communication cost is one of the major bottlenecks in FL. A widely used approach to reduce communication burden is through compression. Various compressors [48, 35, 45, 25] have been studied in the literature. Next, model-heterogeneous FL is also studied to solve the device heterogeneity issue. There are two main categories of heterogeneous model methods: knowledge-distillation (KD) [34, 22, 9] based and partial-training (PT) [11, 6, 1] based. KD-based methods treat the server model as the teacher and the clients' model as the student. In contrast, PT-based methods select a subset of the server model for each client, and in the aggregation phase, the server simply aggregates the updated sub-models.

Bilevel Optimization. Bilevel problem is a type of two-layer optimization problem [49]. In the machine learning community, Hyper-parameter Optimization problems [28, 7, 4, 12] are one of the mostly studied bilevel problems. Early bilevel optimization algorithms solved the lower problem exactly to update the upper variable. Recently, researchers developed algorithms that solve the lower problem approximately with a fixed number of steps and use the back-propagation technique to compute the approximate hyper-gradient [13, 36, 14, 40, 43]. More recently, single-loop algorithms [16, 18, 23, 30, 26, 8, 51, 31, 10, 20, 21] based on alternative updates of lower and upper level problems are proposed. However, less work studies bilevel problems under the federated setting. [46, 33, 32] studies a type of federated bilevel problems where the lower level problems are federated. In particular, [33] solves the noisy label problem in FL. Note that our FedSep is formulated as a type of federated bilevel problems, however, our problem is different from that in [46, 33]. Firstly, our problem has a unique lower problem for each client, moreover, FedSep impose unique constraints not covered by the existing methods. Please refer to Section B in Appendix for more related works.

3 Separating Communication and Learning in Federated Learning

In this section, we propose FedSep , a novel two-layer Federated Learning Framework. As shown in Figure 1, FedSep has the one-server-multiple-clients structure as in the classical Federated learning framework, we assume M clients in the training. The difference between the FedSep and classical FL framework lies in the client. Different from the classical FL, FedSep includes two separated layers on clients: a communication layer and a learning layer. The two layers have different responsibilities. As the name shows, the learning layer is responsible of finishing the learning task, e.g. fitting a neural network over a dataset. We assume that the learning layer on client m ∈ [ M ] has parameter y ( m ) ∈ R d ( m ) . Note that the choice of y ( m ) should be adapted to the specific setting of each client, such as the data distribution, hardware resources etc. . Next, the communication layer performs communication with the server (other clients), which includes the upload and download operations. We assume the communication layer on client m has parameter x ∈ R p . Note that the parameter of communication layer has identical formulation for all clients. Finally, the communication layer and the learning layer are connected through the encode and decode operations.

To model the various potential relationship between the communication and learning layer, we abstract the decode operation as solving a minimization problem, and the encode operation be its reverse operation. As a result, we have the following bilevel formulation for the FedSep framework.

In Eq. (1), M is the number of clients; f ( m ) ( y ) , m ∈ [ M ] denotes the local problem solved by the client m ; g ( m ) ( x, y ) is the minimization problem solved in the decode operation by the client m ; x is the parameter of the communication layer; y ( m ) denotes the parameter of the learning layer on the client m .

Next, we propose a three-stage algorithm to solve Eq. (1). As shown by Algorithm 1, we perform T global steps, and at each step t ∈ T , we randomly choose a subset of clients M t to perform the training and then the server performs aggregation. The Client training is divided into three stages: Decode stage (lines 7-12), Learning stage (lines 14-17), and Encode Stage (line 19).

Decode Stage. In Eq. (1), the decode operation solves the lower optimization problem g ( m ) ( x, y ) to obtain y ( m ) x . We denote the decode operator on the m th client by Dec ( m ) {·} , thus we have: y ( m ) x = Dec ( m ) { x } . In Algorithm 1, we solve g ( m ) ( x, y ) approximately with I dec steps of stochastic gradient

descent (Line 8-11) and use the output as an approximation of y ( m ) x , i.e. . y ( m ) I dec = ˜ Dec ( m ) ( x ( m ) ) ≈ y ( m ) x , where ˜ Dec ( m ) ( · ) represents the approximation of the exact decoder Dec ( m ) {·} .

Learning Stage. Next, we perform I steps of stochastic gradient descent to solve the local learning problem f ( m ) ( y ) (Line 14 - 17). This stage is similar to the local gradients in classical FL framework. Note we get ˆ y ( m ) I as the updated learning parameter y ( m ) , more formally, we use ∆ˆ y ( m ) = ˆ y ( m ) I -ˆ y ( m ) 0 = ˆ y ( m ) I -y ( m ) I dec to represent the update in the learning stage.

Encode Stage. After the learning stage, we get the update of the local learning problem ∆ˆ y ( m ) . Suppose we denote the encode operator Enc ( m ) {·} , then we get the update of the communication parameter as ∆ˆ x ( m ) = Enc ( m ) (∆ˆ y ( m ) ) (Line 19). In particular, we choose the following encode operator:

where y ( m ) x = Dec ( m ) ( x ) is the output of the exact decode operation and the second equality can be derived following the implicit function theorem under mild assumptions [16]. To reduce the computation complexity, we use Neumann series to approximate the inverse operation in Eq. (2). More specifically, we have the following approximation: ( ∇ yy g ( m ) ( x, y ) ) -1 ≈ τ ∑ Q q =0 ( I -τ ∇ yy g ( m ) ( x, y )) q , where Q and τ are some constants. Since in the decode stage, we use y ( m ) x ≈ y ( m ) I dec , we get a stochastic approximation of Eq. (2) as:

Remark 1 . We use Eq. (2) as the encode operator due to the following fact about the hyper-gradient:

Note that suppose ∆ˆ y ( m ) = -∇ f ( m ) ( y ( m ) x ) , which means the learning layer performs one step of gradient descent with learning rate 1 in the learning stage of Algorithm 1, then we have: ∇ h ( m ) ( x ) =

Enc { ∆ˆ y ( m ) } = ∆ x ( m ) . Therefore, we update the communication parameter x such that it optimize the overall problem ∇ h ( m ) ( x ) .

Server Aggregation. Finally, the server aggregates the local updates of communication parameter ∆ˆ x ( m ) , m ∈ M t to get the new communication parameter as shown in Line 21 of Algorithm 1.

Difference with existing bilevel algorithms. Bilevel optimization problems are widely studied in the literature [16, 23]. However, FedSep cannot directly apply existing algorithms. In a standard bilevel algorithm, each step of update to the upper level variable requires (approximately) solving of the corresponding lower level problem. However, in each epoch of FedSep, we only solve the lower level problem once and perform multiple steps of update to the upper level variable.

3.1 Convergence Analysis

In this section, we provide the convergence analysis of Algorithm 1. Before we state the convergence result, we first make the following assumptions.

Assumption 3.1. Function f ( m ) ( y ) is possibly non-convex and g ( m ) ( x, y ) is µ -strongly convex w.r.t y for any given x .

Assumption 3.2. Function f ( m ) ( y ) is L -smooth and has C f -bounded gradient.

Assumption 3.3. Function g ( m ) ( x, y ) is L -smooth; ∇ xy g ( m ) ( x, y ) and ∇ y 2 g ( m ) ( x, y ) are Lipschitz continuous with constants L xy and L y 2 , respectively.

Assumption 3.4. We have unbiased stochastic first-order and second-order gradient oracle with bounded variance.

Note that Assumption 3.1-Assumption 3.3 are standard assumptions used to analyze bilevel problems [23, 16]. Assumption 3.4 is standard in analyzing stochastic optimization problems. For a full version of Assumption 3.4, please refer to Assumption C.1 in the Appendix.

In the learning stage of Algorithm 1, we perform I steps of updates to optimize the learning problem, and the update has the form of: ∆ˆ y ( m ) = ˆ y ( m ) I -ˆ y ( m ) 0 = ∑ I -1 i =0 -η ∇ f ( m ) (ˆ y ( m ) i ; B ˆ y ) , therefore we have: ∆ˆ x ( m ) t = ˜ Enc ( m ) { ∆ˆ y ( m ) } = ∑ I -1 i =0 η ˜ Enc ( m ) {-∇ f ( m ) (ˆ y ( m ) i ; B ˆ y ) } . We denote ∆ˆ x ( m ) t,i = ˜ Enc ( m ) {∇ f ( m ) (ˆ y ( m ) i ; B ˆ y ) } . Then the main effort of the proof is bounding the estimation error of ∆ˆ x ( m ) t,i to the hyper-gradient ∇ h ( m ) ( x ) .

More specifically, we first show that ∆ˆ x ( m ) t,i estimates a term µ ( m ) i with bounded variance and bias as stated in the following proposition:

Proposition 3.5. Suppose Assumptions 3.1-3.3 and 3.4 hold and τ < 1 L , we have ∥ E ξ [∆ˆ x ( m ) t,i ] -µ ( m ) i ∥ ≤ G 1 , where µ ( m ) i = -∇ xy g ( m ) ( x, y ( m ) I dec ) ( ∇ yy g ( m ) ( x, y ( m ) I dec ) ) -1 ∇ f ( m ) (ˆ y ( m ) i ) , and E ∥ ∆ˆ x ( m ) t,i -E ξ [∆ˆ x ( m ) t,i ] ∥ 2 ≤ G 2 2 . G 1 and G 2 are some constants related to Q in Eq. (3) and mini-batch size.

Please refer to Proposition C.3 for the specific form of the constants in Proposition 3.5. Next, we have that µ ( m ) i ≈ ∇ h ( m ) ( x ) as shown in the following proposition:

Proposition 3.6. Suppose Assumptions 3.2 and 3.3 hold, the following statements hold:

h ( m ) ( x ) is Lipschitz continuous in x with constant ¯ L i.e., for any given x 1 , x 2 ∈ R p , we have ∥∇ h ( m ) ( x 2 ) -∇ h ( m ) ( x 1 ) ∥ 2 ≤ ¯ L 2 ∥ x 2 -x 1 ∥ , where ¯ L = O ( κ 3 ) .

where we denote the condition number as κ = L/µ .

Please refer to Proposition C.2 for the proof. Proposition 3.6 shows that µ ( m ) i estimates the true hyper-gradient ∇ h ( m ) ( x ) with two types of errors: the estimation error of the decode operation ( ∥ y ( m ) I dec -y ( m ) x ∥ 2 ) and the drift of the learning process ( ∥ ˆ y ( m ) i -y ( m ) x ∥ 2 ), where Lemma C.4 and

Lemma C.5 show bounds for these two errors. Furthermore, Proposition 3.6.b) shows that h ( m ) ( x ) is smooth. Next, we are ready to show the convergence of Algorithm 1:

Theorem 3.7. Suppose Assumptions 3.1-3.4 hold, and we run T iterations of Algorithm 1, with the learning rates satisfy γ < 1 L , ηη g < 1 2 I ¯ L , then we have:

where the expectation E ξ is w.r.t the noise of stochastic gradient. b x , b y denotes the batch size, and G 1 , G 2 and C 0 are some constants.

Please refer to Theorem C.8 in the Appendix for the proof. We can further control the noise terms in Theorem 3.7 by choosing the learning rates carefully:

Corollary 3.8. Suppose we choose the learning rates as γ = min( 1 , ( 1 ) 1 / 2 ) , η

where ˜ G = κ 2 (1 -τµ ) 2( Q +1) + κ 4 (1 -µγ ) I dec , C η and C γ are some constants.

To reach an ϵ stationary point, we need to run Algorithm 1 with T = O ( κ 5 ϵ -2 ) number of iterations, furthermore, we need Q = O ( κ log( κ ϵ )) , I dec = O ( κ log( κ ϵ )) . Note that this matches the iteration complexity of standard FL algorithms [50] up to some logarithm factors ( Q and I dec ). Note that Corollary 3.8 shows that the following term converges to 0: E ∥∇ h ( x t ) ∥ 2 + 1 2 I ∑ I i =1 E ∥ E ξ [ ¯ ∆ˆ x t,i ] ∥ 2 . In fact, for the first term, it shows that x t reaches to a stationary point of the bilevel problem h ( x ) , meanwhile, since we have:

Therefore, if ∥∇ xy g ( m ) ( x, y ) ∥ is lower-bounded and ∥∇ yy g ( m ) ( x, y ) ∥ is upper-bounded, we also get the learning parameter ˆ y ( m ) i converges to the stationary point of the local learning problem f ( m ) ( y ) .

4 Application of FedSep to Real-world FL Problems

In FedSep, the separation of the communication layer and the learning layer makes it able to solve various challenges in Federated Learning. In this section, we apply FedSep to solve two challenging problems in Federated Learning: communication-efficient FL and model-heterogeneous FL.

4.1 Communication-efficient Federated Learning

Communication-cost is one of the major bottlenecks in Federated Learning due to slow connections between clients and the server. In Federated Learning, compression is commonly employed to mitigate communication costs, where clients compress the local updates before transferring to the server. A common compressor is choosing the Top-K coordinates, however, this approach is only good at reducing the upload communication cost. Since clients have different Top-K coordinates, the aggregated updates are often dense. More complicated approaches can save both upload and download communication cost, such as the Count-sketch based compressor [42]. In fact, we can develop a simple yet effective approach to reduce the communication cost based on FedSep.

In our FedSep framework, suppose we have the learning parameter θ ∈ R d , and the communication parameter ω ∈ R p . We choose p ≪ d to have a communication efficient federated learning algorithm. More specifically, we consider the following formulation:

where D ( m ) tr denotes the training distribution of the m th client; L ( · ) denotes the loss function. In particular, S ( m ) ∈ R p × d ( p ≪ d ) is a random sketch matrix whose coordinates are sampled from Gaussian distribution. We choose the quadratic optimization problem (LASSO) as the decode function.

Eq. (4) is a special case of Eq. (1): θ corresponds to the learning parameter y , ω corresponds to the communication parameter x , L ( θ ; D ( m ) tr ) corresponds to the learning problem f ( m ) ( y ) and the decoding problem is a LASSO problem. We can solve it through Algorithm 1.

The LASSO problem of the decode operation involves a non-smooth L 1 regularization term and we solve it with the standard proximal gradient method [3]. For the encode operation, we have:

where I is the identity matrix and U = Diag { I γβ ( θ ( m ) ω + γ ( S ( m ) ) T ( ω -S ( m ) θ ( m ) ω )) } where I γβ ( · ) is an indicator operator, which outputs 1 if the absolute value of the input is greater than γβ . Please refer to Appendix A.1 for more details of the proximal gradient method and the encode operator. Remark 2 . If we set β = 0 in Eq. (4), i.e. we remove the sparsity constraints, both encode and decode operators have explicit solutions. More specifically, we have the decode operator Dec ( m ) {·} := ( ( S ( m ) ) T S ( m ) ) -1 ( S ( m ) ) T and the encode operator Enc ( m ) {·} := S ( m ) ( ( S ( m ) ) T S ( m ) ) -1 . If we choose S ( m ) = S for all m ∈ [ M ] , we get a linear compressor.

4.2 Model-Heterogeneous Federated Learning

In practical Federated Learning applications, the scale of the model is inherently limited by the on-device resources of the participating clients, which often exhibit significant diversity. As a result, we can choose different scale of models based on the available resources of each individual client. One widely-used approach is the sub-model extraction, i.e. each client selects a part of the server model to perform local training. Different strategies can be used to do extraction: fixed [11, 2], random [6] and roll [1]. In contrast, we provide a data-dependent approach to select the sub-models. As shown by the following formulation:

where D ( m ) tr denotes the training distribution; L ( · ) denotes the loss function and θ and ω are the learning parameter. Note that ω denotes the parameter of the full model and θ is the parameter of the sub-model. In particular, we denote a mask vector a ( m ) ω , whose value is from { 0 , 1 } . ⊙ represents the coordinate-wise multiplication.

For the decode operation, we have θ ( m ) ω = a ( m ) ω ⊙ ω . So we first need to find an optimal mask a ( m ) ω , instead of solving the complicated integer programming as in Eq. (6), we solve the following relaxed continuous problem:

Eq. (7) includes two parts. L ( a ⊙ ω ; D ( m ) val ) measures the loss of the extracted sub-model over D ( m ) val . The second part R ( T ( a ) , p ( m ) T tol ) is called resource loss [15]. T tol denotes FLOPS of

OOT

100

100

90

Test Accuracy

90

Test Accuracy

80

60

Test Accuracy

70

Test Accuracy

80

60

80

70

60

0.5

fedSep

Uncomgressed

op-K

Count-Sketch

1.0

logo(Compression Rate)

1.5

2.0

2.5

20

40

0

0

100

Communication Rounds

200

300

p = 500

000 = d

p = 5000

p = 10000

p = 20000

400

50

30

0.5

Fed5ep

Top-K

Count-Sketch

Uncompressed

1.0

log1o(Compression Rate)

1.5

2.0

2.5

40

20

D

100

Communication Rounds

200

300

p = 500

p = 2000

p = 5000

p = 10000

p = 20000

400

the server model and T ( a ) denotes FLOPS of the sub-model. Therefore, p ( m ) ∈ [0 , 1] controls the size of the sub-model. Following [15], we choose R ( · ) as: R ( T ( a ) , p ( m ) T tol ) = log( max ( T ( a ) , p ( m ) T tol ) /p ( m ) T tol ) . In summary, we choose the 'best' sub-model which keeps the loss value small and meanwhile, satisfies the FLOPS constraint. Note that this is in contrast to the data-independent sub-model selection methods [11, 6, 1].

Next for the encode operator Enc {·} , we follow the definition of Eq. (2) to have:

Where the last equality follows from the chain rule. Finally, combine Eq. (7) and Eq. (8), we can solve Eq. (6) with Algorithm 1.

Remark 3 . For the encode operator Eq. (8), we have Enc ( m ) ( · ) ≈ -Diag { a ( m ) ω } by omitting the second term in Eq. (8), where ∆ θ denotes the updates to the sub-model θ . As shown in Line 21 of Algorithm 1, we have the update of the server model be ∆ ω = 1 |M t | ∑ m ∈M t η g Diag { a ( m ) ω } ∆ θ . In other words, the server simply aggregates the updates of the sub-model, this is the aggregation rule widely used in the partial training literature [11, 6, 1].

5 Numerical Experiments

In this section, we perform numerical experiments to validate the efficacy of FedSep in solving communication-efficient FL and model-heterogeneous FL as discussed in Section 4. The code is written with Pytorch [39], and the Federated Learning environment is simulated via Pytorch.Distributed Package. We used servers with AMD EPYC 7763 64-core CPU and 8 NVIDIA V100 GPUs to run our experiments.

5.1 Experiments for Communication-Efficient Federated Learning

In this section, we provide numerical experiments for the communication-efficient FL task discussed in Section 4.1. We first introduce the experimental settings.

Dataset. We consider MNIST [29] and CIFAR-10 [27] in our experiments. We create the federated version of these datasets by evenly distributing the training samples among 10 clients. We consider an I.I.D setting and a Non-I.I.D setting. For the I.I.D setting, each client has data samples from all 10 classes, while for the Non-I.I.D setting, each client only has data samples from 2 classes.

Model. For experiments related to MNIST, we use a four-layer convolutional neural network. The first three layers contain 32 kernels of size 3 × 3 and the fourth layer contains 32 kernels of size 2 × 2. For the experiments related to CIFAR-10, we increase the number of kernels to 64. The total number of parameters of the model is around 10 5 . For the sketch matrix S ( m ) of our FedSep, we let all clients choose the same sketch matrix, and we test the dimension of the communication parameter p ∈ { 10 , 100 , 500 , 1000 , 2000 , 5000 , 10000 , 20000 } .

Baselines and Hyper-parameters. For baselines, we first compare with the uncompressed baseline (the communication parameter and the learning parameter are in the same dimension); Next we compare with the local Top-K compressor (with error feedback [44]) and the Count-Sketch compressor [42]. Please refer to the Appendix for the hyper-parameter selection of FedSep and baselines.

The experimental results are summarized in Figure 2 for MNIST (Figure 3 in the Appendix for CIFAR10). As shown by the figures, FedSep outperforms other baselines. In particular, our FedSep achieves much better performance in the high-compression-rate regime and can get similar performance as other baselines in the low-compression-rate regime. Please refer to the Appendix for more ablation studies.

5.2 Experiments for Model-Heterogeneous Federated Learning

In this section, we provide numerical experiments for the task of a model-heterogeneous FL task discussed in Section 4.2. First, we introduce the experimental settings.

Dataset. We consider CIFAR-10 and CIFAR-100 [27] in our experiments. We create Federated datasets by evenly distributing images among clients; we have 100 clients in our experiments. We create data heterogeneity by controlling the number of classes K a client can have [1]. For CIFAR-10, we test K ∈ { 2 , 4 , 5 , 8 } , while for CIFAR-100, we test K ∈ { 5 , 10 , 20 , 50 } . Note that smaller values of K mean higher heterogeneity of the data. For FedSep, the validation and training set are the same.

Model. We choose ResNet-18 [17] in the experiments. Following the setting in [1, 11], we replace the batch normalization layer with static batch normalization and add a scalar module after each convolution layer. Instead of using the coordinate mask as defined in Eq. (6), we select kernels in each convolutional layer of ResNet-18; furthermore, we parameterize the mask a with a neural network HN ( ϕ ) [15], i.e. a = HN ( ϕ ) ; finally, we reuse the masks and only update the masks every two epochs, this stabilizes the training. This reduces the dimension of a that we need to optimize. In experiments, the size of the submodel p (Eq. (6)) of a client is randomly chosen from { 1 , 0 . 5 , 0 . 25 , 0 . 125 , 0 . 0625 } , where the ratio p is w.r.t the server model. For our FedSep, after optimizing Eq. (7), we select the top p percent kernels of each convolutional layer by the value of the leaned mask a , and then we only use the selected kernels to perform the training. In the encode operation, we only evaluate the first term of Eq. (8) as stated in Remark 3.

Baselines and Hyper-parameters. We compare with both state-of-the-art Knowledge Distillationbased methods: FedDF [34], DS-FL [22] and Fed-ET [9], and Partial Training Based methods: HeteroFL [11], ZeroFL [41], Federated Dropout [6], FedDST [5], Flash [2] and FedRolex [1]. Fjord [19] gets similar performance as HeteroFL, so we do not include it in the comparision. For ZeroFL, FedDST and Flash, we consider their heterogeneous-model version by varying the compression rate among clients.

The experimental results are summarized in Table 1. As shown in the table, our FedSep gets comparable performance with the Knowledge Distillation based methods, which needs additional public data, furthermore, FedSep outperforms the partial training based methods, including the state-of-the-art FedRolex [1] method. This result shows the effectiveness of the adaptive sub-model extraction strategy used by our FedSep. Our method chooses the most important part of the model

at each step for clients, thus our method converges faster than other data-independent methods. For more experimental results, please refer to the Appendix.

6 Conclusion

In this work, we propose a novel federated learning framework, i.e. FedSep with separated communication and learning components. Based on the observation that communication and learning set opposite requirements to the parameter, we let clients to have two layers: a communication layer and a learning layer. We formulate the decode operation from the communication layer to the learning layer as solving a minimization problem, therefore, FedSep has a bilevel structure. We propose an efficient algorithm to solve this bilevel problem and also prove that the algorithms converge to a stationary point with rate O ( ϵ -2 ) . Finally, we apply FedSep to solve the communication-efficient FL task and the model heterogeneous FL task. Numerical experiments show the superior performance of our FedSep over various baselines.

References

S. Alam, L. Liu, M. Yan, and M. Zhang. Fedrolex: Model-heterogeneous federated learning with rolling sub-model extraction. arXiv preprint arXiv:2212.01548 , 2022.

S. Babakniya, S. Kundu, S. Prakash, Y. Niu, and S. Avestimehr. Federated sparse training: Lottery aware model compression for resource constrained edge. arXiv preprint arXiv:2208.13092 , 2022.

A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences , 2(1):183-202, 2009.

Y. Bengio. Gradient-based optimization of hyperparameters. Neural computation , 12(8):18891900, 2000.

S. Bibikar, H. Vikalo, Z. Wang, and X. Chen. Federated dynamic sparse training: Computing less, communicating less, yet learning better. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 6080-6088, 2022.

S. Caldas, J. Koneˇ cny, H. B. McMahan, and A. Talwalkar. Expanding the reach of federated learning by reducing client resource requirements. arXiv preprint arXiv:1812.07210 , 2018.

D. Chen and M. T. Hagan. Optimal use of regularization and cross-validation in neural network modeling. In IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No. 99CH36339) , volume 2, pages 1275-1280. IEEE, 1999.

T. Chen, Y. Sun, and W. Yin. A single-timescale stochastic bilevel optimization method. arXiv preprint arXiv:2102.04671 , 2021.

Y. J. Cho, A. Manoel, G. Joshi, R. Sim, and D. Dimitriadis. Heterogeneous ensemble knowledge transfer for training large models in federated learning. arXiv preprint arXiv:2204.12703 , 2022.

M. Dagréou, P. Ablin, S. Vaiter, and T. Moreau. A framework for bilevel optimization that enables stochastic and global variance reduction algorithms. arXiv preprint arXiv:2201.13409 , 2022.

E. Diao, J. Ding, and V. Tarokh. Heterofl: Computation and communication efficient federated learning for heterogeneous clients. arXiv preprint arXiv:2010.01264 , 2020.

C. B. Do, C.-S. Foo, and A. Y. Ng. Efficient multiple hyperparameter learning for log-linear models. In NIPS , volume 2007, pages 377-384. Citeseer, 2007.

J. Domke. Generic methods for optimization-based modeling. In Artificial Intelligence and Statistics , pages 318-326. PMLR, 2012.

L. Franceschi, M. Donini, P. Frasconi, and M. Pontil. Forward and reverse gradient-based hyperparameter optimization. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 1165-1173. JMLR. org, 2017.

S. Gao, F. Huang, W. Cai, and H. Huang. Network pruning via performance maximization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9270-9280, 2021.

S. Ghadimi and M. Wang. Approximation methods for bilevel programming. arXiv preprint arXiv:1802.02246 , 2018.

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770778, 2016.

M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A two-timescale framework for bilevel optimization: Complexity analysis and application to actor-critic. arXiv preprint arXiv:2007.05170 , 2020.

S. Horvath, S. Laskaridis, M. Almeida, I. Leontiadis, S. Venieris, and N. Lane. Fjord: Fair and accurate federated learning under heterogeneous targets with ordered dropout. Advances in Neural Information Processing Systems , 34:12876-12889, 2021.

F. Huang and H. Huang. Biadam: Fast adaptive bilevel optimization methods. arXiv preprint arXiv:2106.11396 , 2021.

F. Huang and H. Huang. Enhanced bilevel optimization via bregman distance. arXiv preprint arXiv:2107.12301 , 2021.

S. Itahara, T. Nishio, Y. Koda, M. Morikura, and K. Yamamoto. Distillation-based semisupervised federated learning for communication-efficient collaborative training with non-iid private data. IEEE Transactions on Mobile Computing , 22(1):191-205, 2021.

K. Ji, J. Yang, and Y . Liang. Provably faster algorithms for bilevel optimization and applications to meta-learning. arXiv preprint arXiv:2010.07962 , 2020.

S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh. Scaffold: Stochastic controlled averaging for on-device federated learning. arXiv preprint arXiv:1910.06378 , 2019.

S. P. Karimireddy, Q. Rebjock, S. Stich, and M. Jaggi. Error feedback fixes signsgd and other gradient compression schemes. In International Conference on Machine Learning , pages 3252-3261. PMLR, 2019.

P. Khanduri, S. Zeng, M. Hong, H.-T. Wai, Z. Wang, and Z. Yang. A near-optimal algorithm for stochastic bilevel optimization via double-momentum. arXiv preprint arXiv:2102.07367 , 2021.

A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.

J. Larsen, L. K. Hansen, C. Svarer, and M. Ohlsson. Design and regularization of neural networks: the optimal use of a validation set. In Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop , pages 62-71. IEEE, 1996.

Y. LeCun, C. Cortes, and C. Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yann. lecun. com/exdb/mnist , 2, 2010.

J. Li, B. Gu, and H. Huang. Improved bilevel model: Fast and optimal algorithm with theoretical guarantee. arXiv preprint arXiv:2009.00690 , 2020.

J. Li, B. Gu, and H. Huang. A fully single loop algorithm for bilevel optimization without hessian inverse. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 36, pages 7426-7434, 2022.

J. Li, F. Huang, and H. Huang. Communication-efficient federated bilevel optimization with local and global lower level problems. arXiv preprint arXiv:2302.06701 , 2023.

J. Li, J. Pei, and H. Huang. Communication-efficient robust federated learning with noisy labels. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 914-924, 2022.

T. Lin, L. Kong, S. U. Stich, and M. Jaggi. Ensemble distillation for robust model fusion in federated learning. Advances in Neural Information Processing Systems , 33:2351-2363, 2020.

Y. Lin, S. Han, H. Mao, Y. Wang, and W. J. Dally. Deep gradient compression: Reducing the communication bandwidth for distributed training. arXiv preprint arXiv:1712.01887 , 2017.

D. Maclaurin, D. Duvenaud, and R. Adams. Gradient-based hyperparameter optimization through reversible learning. In International Conference on Machine Learning , pages 21132122, 2015.

B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics , pages 1273-1282. PMLR, 2017.

K. Nandakumar, N. Ratha, S. Pankanti, and S. Halevi. Towards deep neural network training on encrypted data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops , pages 0-0, 2019.

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems , pages 8024-8035, 2019.

F. Pedregosa. Hyperparameter optimization with approximate gradient. arXiv preprint arXiv:1602.02355 , 2016.

X. Qiu, J. Fernandez-Marques, P. P. Gusmao, Y. Gao, T. Parcollet, and N. D. Lane. Zerofl: Efficient on-device training for federated learning with local sparsity. arXiv preprint arXiv:2208.02507 , 2022.

D. Rothchild, A. Panda, E. Ullah, N. Ivkin, I. Stoica, V . Braverman, J. Gonzalez, and R. Arora. Fetchsgd: Communication-efficient federated learning with sketching. In International Conference on Machine Learning , pages 8253-8265. PMLR, 2020.

A. Shaban, C.-A. Cheng, N. Hatch, and B. Boots. Truncated back-propagation for bilevel optimization. arXiv preprint arXiv:1810.10667 , 2018.

S. U. Stich. Local sgd converges fast and communicates little. arXiv preprint arXiv:1805.09767 , 2018.

S. U. Stich, J.-B. Cordonnier, and M. Jaggi. Sparsified sgd with memory. arXiv preprint arXiv:1809.07599 , 2018.

D. A. Tarzanagh, M. Li, C. Thrampoulidis, and S. Oymak. Fednest: Federated bilevel, minimax, and compositional optimization. arXiv preprint arXiv:2205.02215 , 2022.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

W. Wen, C. Xu, F. Yan, C. Wu, Y. Wang, Y. Chen, and H. Li. Terngrad: Ternary gradients to reduce communication in distributed deep learning. arXiv preprint arXiv:1705.07878 , 2017.

R. A. Willoughby. Solutions of ill-posed problems (an tikhonov and vy arsenin). SIAM Review , 21(2):266, 1979.

B. Woodworth. The minimax complexity of distributed optimization. arXiv preprint arXiv:2109.00534 , 2021.

J. Yang, K. Ji, and Y . Liang. Provably faster algorithms for bilevel optimization. arXiv preprint arXiv:2106.04692 , 2021.

Peer reviews:
comments: SUMMARY:
This paper proposes a two-layer federated learning (FL) framework (FedSep) by separating the communication and learning parameters. By a bilevel optimization formulation, FedSep enjoys a convergence guarantee. In addition, two settings, communication-efficient FL and model-heterogeneous FL, are solved in FedSep framework.

STRENGTHS:
1. FedSep is an interesting and novel setting.
2. Theoretical convergence analysis is provided, showing the sublinear rate.
3. FedSep is applied to communication-efficient FL and model-heterogeneous FL.

WEAKNESSES:
I have the following concerns and questions:

1. The convergence rate is sublinear, but it does not show any speedup in terms of the clients' number and local steps. So it does not exactly match the convergence of standard FL algorithms. Any possible improvement? Or the statement in the paper should be modified accordingly (e.g., line 75).

2. In the model-heterogeneous FL, the validation data set is also used in the training process both for problem formulation and experiments. I believe this is a concern and unfair for the direct comparison with other algorithms

QUESTIONS:
see above

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-28

comments: SUMMARY:
The authors proposed a two-layer federated learning framework called FedSep, with one layer for communication and another layer for learning. The two layers are connected through decode/encode operations. Furthermore, the authors proposed an efficient algorithm to solve FedSep by treating it as a bilevel optimization problem, and showed convergence results which match those of the standard framework. The FedSep can incorporate Communication-Efficient FL and Heterogeneous-Model FL.

STRENGTHS:
(S1) The proposed framework, FedSep, provided a general framework for resolving the tug-of-war by separating the communication and learning of federated learning into two layers. 

(S2) The algorithm was explained in detail and the theoretical results are solid. The experiments results supports the claims of the authors well.

WEAKNESSES:
(W) The framework rely on the analyticity and strong convexity of the second level problem, which may not be true in many cases. For example, for problem (equation 4) formulated in Sec 4.1, the analyticity and strong convexity does not seem to hold. Although I still believe some convergence results can be derived.

QUESTIONS:
(Q) The authors claimed that Theorem 3.7 implies that $\mathbb{E}||\nabla h(x_t)||^2 + \frac{1}{2I}\sum_{i=1}^I \mathbb{E}||\mathbb{E}_{\xi}[\bar{\Delta} \hat{x}_t, i ]||$ converges to zero. Because the right-hand side of the inequality in Theorem 3.7 contains constant, this claim is not straightforward to me, can the authors elaborate?

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-04

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper asserts that the tasks of communication and learning are at odds in federated learning. As such, a new approach is suggested that separates these tasks. The approach comprises an encode-decode operation, where decoding is cast as an optimization problem. The overall learning task is formulated as a bilevel optimization problem. Standard first-order gradient-based approaches are then employed to solve the bilevel optimization problem. The structure of the proposed algorithm mirrors that of typical FL algorithms. A convergence bound is proven for this algorithm. Some applications pertaining to communication-efficiency and model-heterogeneity in FL are also discussed.\n\nSTRENGTHS:\n- The particular \"separation\" framework suggested in this paper appears to be novel; I have not seen anything exactly like this in earlier FL work.\n\n- One advantage of the proposed bilevel approach is that each agent can train a separate local model $y^{(m)}$ of its own. This allows for personalization in the face of data-heterogeneity.\n\nWEAKNESSES:\nI have several major concerns ranging from the motivation to the utility of the proposed approach. Let me elaborate on them below. \n\n- The entire paper is based on the premise that communication and learning are conflicting goals in federated learning. However, this statement is never formalized at any point in the paper. The only discussion regarding the tension between communication and learning shows up in lines 25-30, which don't convey anything concrete.  \n\n- Continuing with the above point, I also missed the motivation for the specific bilevel formulation. By now, there are several approaches that guarantee communication-efficiency in RL (see Refs [R1]-[R5] below) using techniques ranging from quantization, sparsification, compressed sensing, etc. It stands to reason that compressing information about models/gradients will naturally come at the cost of performance in learning. So the tension here is evident, and has been well-explored/quantified in the papers I mentioned earlier. In particular, sending less bits of information slackens the rate of convergence, and one can potentially try to investigate the fundamental limits on the rate needed to achieve a desired level of accuracy; see, for instance, [R5]. However, there is no clear discussion of why the FedSep bilevel idea is any significant improvement over any of these MANY existing schemes. \n\n[R1] Federated learning with compression: Unified analysis and sharp guarantees, Haddadpour et al., AISTATS 21\n\n[R2] Linear convergence in federated learning: Tackling client heterogeneity and sparse gradients, Mitra et al., NeuRIPS 21\n\n[R3] Optimizing the communication-accuracy trade-off in federated learning with rate-distortion theory, Mitchell et al., arXiv 2022\n\n[R4] Communication-Efficient Federated Learning through Importance Sampling, Isik et al., arXiv 2023\n\n[R5] Differentially quantized gradient methods, Yin et al., IEEE Transactions on Information Theory, 2022\n\n- Regarding model-heterogeneity, some of the references I alluded to earlier do account for heterogeneous loss functions across agents. Moreover, if the goal is to allow for personalized models, then why not use any of the existing schemes for personalization in FL (of which, there are many)? See, for instance, ideas based on Moreau Envelopes and MAML in [R6] and [R7], and representation learning in [R8]. If one cares about both personalization and communication-efficiency, I can imagine simply using these ideas in tandem (in meaningful ways). \n\n[R6] Personalized Federated Learning with Moreau Envelopes, Dinh et al, NeuRIPS 2020\n\n[R7] Personalized Federated Learning: A Meta-Learning Approach, Fallah et al., NeuRIPS 2020\n\n[R8] Exploiting Shared Representations for Personalized Federated Learning, Collins et al, ICML 21\n\nThe overarching point I am trying to make here is that each of the key considerations in FL that the authors allude to (communication-efficiency, heterogeneity, personalization, etc), have several existing principled algorithmic solutions. It wasn't apparent to me at all why there is a compelling need to depart from these existing approaches.\n\n- In addition to the motivation, several parts of the paper are somewhat vaguely written. For instance, in Eq. (1), the meaning of the object $g^{(m)}$ is not explained. The encoding operator in Eq.~(2) isn't clear to me either. What is this operation and how does it compare with any of the standard encoding techniques (say for instance, standard scalar and vector quantizers, or sparsifying mechanisms like Top-k)? How many bits are needed to perform this encoding? What is the error caused by this encoding? Is this an unbiased encoding mechanism? No intuition is provided at all about any of these crucial points, making it hard to draw meaningful comparisons with existing schemes.\n\n- The difference with existing federated bilevel optimization algorithms in lines 153-157 is also quite terse. The discussion did not come across as anything fundamental. It wasn't clear to me why the bilevel algorithm proposed can't be analyzed by simply adapting ideas from other existing FL bilevel algorithms. \n\n- The main convergence result in Theorem 3.7 is hard to parse. I was left wondering about several key questions: (i) How does the compression scheme (encoding-decoding) affect the rate of convergence? (ii) How does this trade-off compare with the existing known bounds for compression in FL? (iii) How does the effect of heterogeneity in the agents' loss functions manifest in the bounds? Does this dependence match with those known for federated bilevel optimization? \n\nAlso, the iteration complexity seems to have a $\\kappa^5$ dependence on the condition number $\\kappa$. This seems much larger than what one typically obtains. Thus, I am not convinced about the tightness of the bounds either.\n\nQUESTIONS:\nNo questions other than the ones above.\n\nLIMITATIONS:\nI couldn't find a clear discussion of the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper uses bilevel optimization in a federated learning setting. A unique decomposition of communication and learning has been identified, which has applications for reducing communication overhead and supporting heterogeneous models. The theoretical convergence guarantee has been presented and experiments also show the advantage of the proposed method.\n\nSTRENGTHS:\n- The use of bilevel optimization as a decomposition into communication and learning in a FL setting is interesting. \n- The framework is general and supports multiple application scenarios of FL.\n- Theoretical convergence bound is provided.\n\nWEAKNESSES:\n- The use of encoder/decoder structure may incur additional computation compared to common FL algorithms. It would be nice to quantify the amount of such additional computation.\n- The extension of common bilevel optimization algorithms to include multiple update steps (as pointed out at the end of page 4) seems to be somewhat similar to the idea of local updates in FedAvg / local SGD algorithms. It would be nice to identify what are the key technical challenges and novel solution techniques in this extension. \n- There is space for improvement in the experiments (see details below).\n- The writing could be improved to highlight the usefulness of encoder/decoder in the context of FL, in early parts of the paper.\n\nQUESTIONS:\n- What is the additional computation overhead compared to standard FL algorithms such as FedAvg?\n- What are the key technical (mathematical) challenges and novelties?\n- In the experiments in Section 5.1, biased compressors such as top-K usually only works well with error feedback [a]. Is error feedback used in the baseline methods? In addition, are the first and third plots in Figure 2 results at convergence? Theoretically, all methods (when using error feedback in baselines) should converge to the optimal solution when the experiments are run long enough. More explanation is needed on why a much worse accuracy is obtained when the compression rate is high. Is it a matter of hyper parameter tuning? It may be better to plot the accuracy against the amount of communication (e.g., in the number of bytes transmitted, *not* in communication rounds), to compare the accuracies of different methods and different compression rates at the same amount of communication.\n- In Section 5.2, the advantage of having heterogeneous models is not quite clear. \n\n[a] Stich, Sebastian U., and Sai Praneeth Karimireddy. \"The error-feedback framework: Better rates for SGD with delayed gradients and compressed communication.\" arXiv preprint arXiv:1909.05350 (2019).\n\nLIMITATIONS:\nN/A", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-12"}