Tunable Soft Prompts are Messengers in Federated Learning
Authors: Chenhe Dong
Yuexiang Xie
Bolin Ding
Ying Shen
Yaliang Li
Venue: EMNLP 2023 Findings (2023)
Keywords: federated learning

Abstract: Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources, alleviating privacy concerns that arise from directly sharing local data. However, the lack of model privacy protection in FL becomes an unneglectable challenge, especially when people want to federally finetune models based on a proprietary large language model. In this study, we propose a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts. These soft prompts, updated and transmitted between the server and clients, assume the role of the global model parameters and serve as messengers to deliver useful knowledge from the local data and global model. As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL. Extensive experiments show the effectiveness of the proposed approach compared to several baselines. We have released the source code at https://github.com/alibaba/FederatedScope/tree/fedsp/federatedscope/nlp/fedsp.

Tunable Soft Prompts are Messengers in Federated Learning
Chenhe Dong 1:41
Yuexiang Xie Bolin Ding
Ying Shen Yaliang Li
Sun Yat-sen University Alibaba Group
dongchh @aail2.sysu.eduu
(yuexiang.xyx, bolin.ding, yaliang. @aaigglll@@@lllliiiiii @alibaba-inc.com
sheny76 @ mail.sysu..eu.com
Abstract
good use of private and decentralized data can be one of the critical steps, which brings both op- portumities and challenges to federated learning (FL) (Konečný et al., 2016; McMahan et al., 2017; Yang et al., 2019; Kairouz et al., 2021).
Federated learning (FL) enables multiple participants to collaboratively train machine learning models using decentralized data sources, alleviating privacy concerns that arise from directly sharing local data. However, the lack of model privacy protection in FL becomes an unneglectable challenge, especially when people want federally finetune models based on proprietary large language model. this study, we propose novel FL training approach that accomplishes information exchange among participants via tunable soft prompts. These soft prompts, updated and transmitted between the server and clients, assume the role the global model parameters and deliver useful knowledge from the local data and global model. As the global model itself is not required to be shared and the local training is conducted based on an auxiliary model with fewer parameters than the global model, the proposed approach provides protection for the global model while reducing communication and computation costs in FL. Extensive experi- ments show the effectiveness of the proposed approach compared to several baselines. We have released the source code at https: thgi com/ cctmm//iibbbbaeeeerrttttsoooo htee///elss/eeddrrtteeeccpppe//p
The main idea of FL that multiple participants collectively train a global model using their local data, and the updates generated by each partici- pant's local training process are then aggregated together to optimize the global model. On the one hand, FL. offers feasible solution for finetuning LLMs by utilizing multiple data sources without private data leakage. Participants conduct local training processes based on their pri- vate data, and then share the learned knowledge through the exchange of model updates. this way, privacy concerns, raised by directly sharing private data, can be alleviated.
allowed to
as messengers
However, on the other hand, the application of FL can be constrained by concerns regarding model privacy, especially when the finetuning process re- lies on proprictary LLMs. In particular, during each training round, the up-to-date global model would be distributed to participants, which goes against the interests of model owners and might be deemed unacceptable in real-world scenarios.
Such conflicts between protecting data privacy and model privacy are attributed to the training paradigm of FL, which involves sharing model pa- rameters to accomplish the knowledge exchange process for collaboratively training models. To ad- dress this challenge, we propose FEDSP, novel FL training approach that leverages tunable soft prompts to enable the exchange of useful knowl- edge among participants, achieving both data pri- vacy and model privacy protection at the same time.
1 Introduction
Large language models (LLMs) (Radford et al., 2019; Brown et al., 2020; Zhang et al., 2022; Chowdhery et al., 2022; Scao et al., 2022; Zeng al., 2022; OpenAI, 2023; Touvron et al., 2023) have been witnessed incredible progress in the re- cent years, which is tied the support of large amounts of training corpus and computation re- sources. To further promote the development of LLMs and broaden their applications, how to make
Soft prompts can be regarded as some additional tunable parameters plugged into the model, which are updated to capture knowledge from down- stream tasks while keeping the model parameters frozen in parameter- efficiien finetuning (PEFT) al-
'Equal contribution.
"ork done at Alibaba.
*Coreesponding authors.

---

gorithms (Houlsby et al., 2019; Lester al., 2021; Li and Liang, 2021; Hu al., 2021; Zaken al.. 2021). We incorporate these tunable soft prompts in FL. serving as messengers among participants to deliver knowledge learned from local data and contained in the global model.
2023) are usually kept at the cloud server who owns adequate resources for inferring, finetuning, and training. Nowadays, users of LLMs have to send the instructions to the cloud server and wait for the model-generated results, or upload their private data' for finetuning the LLMs on their downstream tasks. The server can also benefit from the user instructions and human feedbacks to continually improve the model performance.
Specificallyy, at the start of FL course, a server broadcasts an auxiliary model (which typically has much fewer parameters than the global model) to participating clients. Then in each training round, the server sends up-to-date tunable soft prompts to selected clients, and these clients combine the received soft prompts with their maintained auxil- iary models. After that, each client performs local training accomplish the process consisting of (i) Global Model Alignment, in which clients update the auxiliary models to align them with the repre- sentative capabilities of the global model, using the up-to-date soft prompts; and (ii) Local Knowledge Capturing, in which clients freeze their auxiliary models and finetune the soft prompts to capture use- ful knowledge from their local data. These updated soft prompts are sent back to the server once the local training is complete. The server is responsible for aggregating and optimizing these soft prompts to drive the next training round.
Such usage of LLMs might bring privacy issues from two different perspectives. For the users of LLMs, they might not be allowed to upload their private data to the server, especially in some sce- narios with highly sensitive information such as health care, so-called data privacy. For the cloud servers (i.e., the model owners), they tend to keep the proprietary language model private to ensure their interests, so-called model privacy, As a result, how to provide protection for both data privacy and model privacy is a practical and challenging problem when training, finetuning, and deploying LLMs in real-world applications.
Federated Learning (FL) (Konečný et al., 2016; McMahan et al., 2017) is one of the considered solutions for alleviating the aforementioned data privacy issue. Different from the centralized train- ing paradigm that needs to gather users' local data for training a global model, FL. proposes to aggre- gate participants' model updates to avoid directly sharing the private data. To be more specific, at each training round, a server broadcasts the up-to- date global model to the participating clients. Each client updates the received global model based on its local data, and then sends the model updates back to the server for performing federated aggre- gation. Formally, the federated aggregation at the 1-th training round can be defined as:
In contrast to the existing FL training paradigm that entails sharing the parameters of the global model, the proposed FEDSP suggests exchanging tunable soft prompts during training rounds, which ensures privacy protection for the global model. Meanwhile, clients are only expected to update the auxiliary models and soft prompts in the local train- ing process, which mitigates both computation and communication overhead compared to federally finetuning of large global models like LLMs.
We conduct series of experiments with two LLMs (GPT2-XL and O.T-1.3B) on seven bench- marking datasets, showing that FEDSP achieves competitive performance compared to baseline methods while significantly reducing the model size by 14.5x/8.5x on GPT2-XL/OPT-133B These experimental results demonstrate the effectiveness and advantages of the proposed idea that using tun- able soft prompts as knowledge messengers in FL..
K
N N
un
(1)
where K is the number of clients, N is the total amount of training data, wy is the updated model parameters of the k-th client, and wy is the aggre- gated global model parameters.
Nevertheless, as the global model is required to be shared in the training process of FL, the model privacy issue has not been well addressed yet, which motivates to make an improvement in providing both data privacy protection and model privacy protection in FL..
2 Preliminary
Because of the huge model size (billions of param- eters), large language models (LLMs) (Chowdh- cry et al., 2022; Zeng et al., 2022; OpenAI, 2023; Touvron et al., 2023: Zhao et al., 2023; Chen et al.,

---

Tunable soft prompts
Model parameters
Shared parameters
Recurrent forward
Trainable
Frozen
Global Model
Global Model
Local Knowledge Capturing
Soft prompts
Server
Alignment
X
Auxiliary Model
L
L
Client
Client
Client
(a) Tunable soft prompts as messengers
(b) A Alternative local training
Figure 1: Overall architecture of the proposed FEDSP.
3 Methodology
ing the probability distribution of the next token. By optimizing these soft prompts over the train- ing datasets via backward propagation, the knowl- edge contained in the data can be captured and condensed into their parameters.
In this section, we describe the details of the pro- posed FEDSP. The intuition behind FEDSP is to adopt some tunable soft prompts to replace the shared global model in the training process of FL. which serves as messengers to deliver useful knowl- edge in the local data and the global model to the server and clients, as introduced in Section 3.1. With the help of these tunable soft prompts, clients are expected perform local training for updating auxiliary model and the soft prompts alterna- tively, and send the updated soft prompts to the servers for sharing the knowledge learned from their local data (more details in Section 3.2). The overall architecture of the proposed FEDSP is illus- trated in Figure! 1.
In this study, we follow the tunable soft prompts proposed by PREFX-TUNING (Li and Liang, 2021). The tunable soft prompts would be updated and transmitted between the server and clients for exchanging useful knowledge. Note that soft prompts should be plugged into model first, and are updated based on the training data while other parameters of this model are kept frozen. For the server that owns the global model, it is straightfor- ward and reasonable to add tunable soft prompts to the global model. However, adding and maintain- ing soft prompts for the clients becomes challeng- ing, since clients could not access the global model due the model privacy issue.
3.1 Tunable Soft Prompts
The existing FL algorithms (Yang et al., 2019; Kairouz et al., 2021) leverage the global model pa- rameters exchange the useful knowledge learned from participants' local data. However, it might raise the model privacy issue when the training pro- cess conducted on a private model, such as fine- tuning a proprictary large language model on down- stream tasks. Inspired by the parameter-efficient finetuning (PEFT) algorithms (Lester et al., 2021; Li and Liang. 2021). 1), we propose to adopt tunable soft prompts to tackle such model privacy issue. These soft prompts are continuous vectors added to the LLMs, serving as instructive contexts to in- fluence the generation process of L.MSs by steer-
In order to tackle such challenge, each client is expected to integrate the tunable soft prompts with auxiliary model and perform alternative local training process. Before we introduce more details of the local training, we first pay more attention to the auxiliary model. In the proposed FEDSP. the auxiliary model should tied to the global model but could not cause model privacy leakage, and had better be a shadow model that has fewer parameters than the global model considering data quantity and computation resources of client be limited in some application scenarios. Motivated by the aforementioned requirements, we construct the auxiliary model by substantially

---

reducing the depth of the global model and apply- ing the cross-layer parameter sharing, inspired by the main idea of Albert (Lan et al.. 2020). Given that most of the LLMs adopt the transformer-base based architecture (Vaswani et al., 2017), such a design of the auxiliary model is suitable for plugging and tuning the soft prompts shared by the server.
process meaningful, we propose to provide a good initialization of the auxiliary model and perform an alternative local training process for the clients.
The different model sizes between the server and clients might lead to the misaligned representation, and further cause the updating of the soft prompts mismatch between the server and clients. Such a mismatch seriously hinders effec- tive knowledge exchange between the server and clients in an FL course.
Initialization
The example illustrated in Figure I(a) shows that, the server owns a global model with total L layers and each layer is concatenated with some prefix prompts, while each client keeps an auxiliary model with only layer and plugged with L-layer prefix prompts. Using the auxiliary model, clients only need to update the tunable soft prompts based on their local data and exchange the updated soft prompts with others to accomplish the FL course. Thus the proposed FEDSP provides privacy pro- tection for the global model as the global model is only maintained by the server and won't be shared. Meanwhile, clients need fewer computation and communication resources for updating the auxil- iary model and soft prompts in the local training process compared to those of training the global model as in previous studies.
To alleviate such a mismatch, the server lever- ages knowledge distillation (KD) (Hinton et al.. 2015) techniques to initialize the auxiliary models before sharing them with clients. The main idea is to align the representations of the student models (i.e., auxiliary models) with the large teacher model (i.e., the global model), which is widely used in pre- vious studies (Cheng al., 2021; Xiao et al., 2023) for representation alignment. Formally, given the last hidden states of the teacher model H and the student model H the loss function of KD can be defined as:
L = MSE(H" W
(2)
In the rest of this section, we describe the local training process of clients in the proposed FEDSP, showing how to deliver useful knowledge via up- dating and exchanging the tunable soft prompts.
where MSE is the mean square error function, WS is a learnable transformation matrix.
Notably, the tunable soft prompts are not added to the global model and auxiliary models during the KD process. And the aforementioned KD process is only performed by the server at the beginning of an FL course, whose computation cost is afford- able. After initialization, the auxiliary models are broadcast to all participating clients and updated by clients according to their local data independently.
3.2 Training Procedure
At the beginning of an FL course, the server first initializes the auxiliary model according to its global model, and broadcasts the generated aux- iliary model to all the participating clients. Then the server sends the up-to-date soft prompts to the selected clients at each training round, while the clients finetune the received soft prompts and also update the auxiliary model accordingly. The up-
Alternative Local Training When receiving the auxiliary models, which have fewer layer numbers compared to the global model, the clients apply the cross-layer parameter sharing (Lan et al., 2020) to imitate the auxiliary models as the global model in the local training process. Each shared layer is concatenated with the corresponding tunable soft prompts. In this way, the tunable soft prompts in the server and clients are served in the same manner in an FL course.
dated soft prompts are uploaded to the server for performing federated aggregation, and then the ag- gregated soft prompts are plugged into the global model and further optimized by the server. Finally, the up-to-date soft prompts are sent clients to start new training round.
Note that compared to the existing FL training paradigms that exchange the global model, in the proposed FEDSP, the models kept at the server (i.e.. the global model) and clients (i.e., the auxiliary models) have different model sizes but are plugged with the same prompts. As a result, to narrow the optimization gap and make the federal training
During the local training process, clients adopt alternative training method to achieve both global model alignment and local knowledge cap- turing, as shown in Figure I (b). To be more spe- cific, firstly clients concatenate the received soft prompts with the local auxiliary models, and fine- tune the parameters of the auxiliary models while

---

Table I: The comparisons between the proposed FEDSP and baselines with GPT2-XL. -L.
Methods
ARC-C
ARC-E
HellaSwag
OpenBookQA
PIQA
RACE
SciQ
ZERO-SHOT
25.1
58.2
40.0
23.0
70.9
33.0
83.2
FINETUNE
30.0
62.9
40.7
30,0
73.2
43.2
92.5
PREFIX-TUNING TUNING
28.2
58.9
40.4
25.6
72.3
38.2
92.9
FEDPROMPT
27.5
59.4
40.7
26.2
71.9
38.3
92.8
PEMPROMPT-SINGLE
20.1
38.3
35.0
12.4
63.9
31.6
72.3
FEDSP (ours)
26.5
61.2
40.9
24.2
71.0
35.2
92.8
w/o KD
17.8
41.4
40.1
13.0
62.2
36.7
84.4
w/o CS
21.0
44.6
40.0
10.6
66.6
37.2
89.2
w/o AT
25.6
56.4
37.1
13.4
69.7
34.3
81.0
freezing the soft prompts. Since these soft prompts have been plugged into the global model and up- dated by the server before being shared, the purpose of freezing soft prompts and updating the auxiliary models is align the representation of auxiliary models and the global model, with the help of the same soft prompts. Such an alignment is a neces- sary step in local training as the misaligned might cause a mismatch of the soft prompts between the server and clients.
4.2 Baselines
We compare the proposed FEDSP with five differ- kinds of baseline methods, including:
ZERO-SHOT,, which directly evaluates the pre-trained language models without updating based on the downstream datasets.
FINETUNE, which finetunes the entire LLMs on the downstream datasets, respectively.
After the global model alignment, clients per- form prompt tunning as those used in PEFT algo- rithms, which implies that clients freeze the aux- iliary models and only finetune the soft prompts for learning useful knowledge from local data, so- called local knowledge capturing. These updated soft prompts sent back to the server, serving messengers to deliver useful knowledge learned from participants' local data.
PREFIX-TUNING (Li and Liang, 2021), which adds tunable soft prompts and only fine- tunes them on the downstream datasets, re- spectively. Note that the parameters LLMs are frozen during the finetuning process.
FEDPROMPT (Zhao et al., 2022), which applies PRENI-TTUNING in the field of FL. Clients need to keep both the global model and the soft prompts and only update and ex- change the soft prompts. Note that the param- eters of the global model are frozen.
4 Experiments
4.1 Models and Datasets
FEDPROMPT-SINGLE. a variant of FED- PROMPT, which reduces the layer number of clients' global model to I to ensure the privacy protection of the global model. This setting is similar that of FEDSP.
We conduct a series of experiments with two popu- lar large language models, i.e., GPT2-XL (Radford al.. 2019) and OPT-1.3B (Zhang et al., 2022). Specifically, both GPT2-XL and OPT-1.3B adopt the transformer-based architecture (Vaswani et al.. 2017). GPT2-XL. has 48 layers and 1.5 billion pa- rameters, and OPT-1.33 1.3 has 24 layers and 1.3 bil- lion parameters. For the benchmarking datasets, we adopt seven ceeestion-answeriie datasets for quan- titative analysis, including ARC-C/E (Clark et al., 2018), HellaSwag (Zellers et al., 2019). Open- BookQA (Mihaylov et al., 2018), PIQA (Bisk et al., 2020), RACE (Lai et al., 2017), and SciQ (Welbl et al., 2017). We use accuracy as the evaluation metric the experiments.
4.3 Implementation Details
The experiments are conducted on eight NVIDIA GeForce RTX 3090 GPUs. We implement the pro- posed approach and baseline methods based on Huggingface Transformers (Wolf al., 2020) and FederatedScope (Xie et al., 2023), and conduct the evaluation with the support of 1m-eval2. Follow- ing previous study (Zhao et al., 2022), we divide

---

Table 2: The comparisons between the proposed FEDSP and baselines with OPT-1.3B.
Methods
ARC-C
ARC-E
HellaSwag
OpenBookQA
PIQA
RACE
SciQ
ZERO-SHOT
23.5
56.9
41.5
23.4
71.5
34.2
84.4
FINETUNE
27.7
61.3
42.7
31.4
75.2
37.0
92.5
PREFIX-TUNING TUNING
27.4
57.7
41.9
27.2
73.2
38.5
89.6
FEDPROMPT
26.8
57.6
41.8
27.4
72.9
38.4
89.8
PEMPROMPT-SINGLE
24.6
54.4
38.3
19.0
68.8
30.4
80.5
FEDSP (ours)
26.8
58.0
42.1
26.0
71.9
36.6
89.9
w/o KD
22.1
53.5
40.8
17.6
68.0
32.2
80.7
w/o CS
24.3
53.5
40.6
18.0
69.3
36.0
76.5
w/o AT'
26.2
56.4
35.1
21.8
71.0
33.2
89.8
each dataset into 10 partitions, and each client owns one of them. A linear decay learning rate sched- uler with warm-up proportion of 0.1 is used in the experiments. We adopt AdamW (Loshchilov and Hutter, 2019) as the optimizer with 31 0.9, 82 0.999, the weight decay is set to 0.01, and the batch size is set 16. The layer number of the auxiliary model used in FEDSP is set to I.
FL algorithms in maintaining the performance com- pared to the central training algorithms. However, there exists noticeable gap between the perfor- mance achieved by FEDPrOMPT and FINETUNE, which is attributed to the fact that the number of tunable parameters FEDPrOMPT much fewer than those in FINETUNE.
Further, FEDPROMPT-SINGLE performs sig- nificantly worse than FEDPROMPT, and even worse than ZERO-SHOT on almost all the adopted datasets. These experimental results further con- firm that the misalignment between the server's 's and clients' models might bring the optimization gap and make the federal training process meaningless, as discussed in Section 3.2.
We conduct the grid search for the optimal hyper- parameters. Specifically, the learning rate is tuned [1e-4, 2e-4, 5e-4), , the training round is tuned (20, 50, 100, 200), and the local training step is tuned in [10, 20). For the methods that adopt prefix-tunning, we set the dimension of the tunable soft prompts to 40. And we conduct knowledge distillation for 5000 training steps with an initial learning rate of 5e-4. Inspired by previous study Li and Liang (2021), we adopt the reparametrization strategy with hidden size of 512 on HellaSwag, RACE, and SciQ datasets.
The proposed FEDSP achieves much better per- formance compared to CAAPPROPPT-SINGGA and is competitive with those of FEDPROMPT and PREFIX-TUNING For example, evaluated on the ARC-C dataset with GPT2-XL/OPT-IL the per- formance of FEDSP significantly better than FEDPROMPT--INGHE with an improvement of 6.4%/2.2%, and is similar to FEDPROMPT within 1% performance gap. These results show the ef- fectiveness and advantage of FEDSP in solving the misalignment issue and using tunable soft prompts as messengers for knowledge delivery. Meanwhile, with the help of these tunable soft prompts, it is worth pointing that the proposed FEDSP pro- vides both privacy protection for local data and global model, and needs fewer computation and communication costs compared to baselines.
4.4 Experimental Results
The comparison results between FEDSP and baseline methods are demonstrated in Table 1 and 2. From these results we can observe that, FINETUNE brings large performance boost the downstream tasks compared to those results achieved by ZERO-SHOT, and costs lots of com- putation resources. Therefore, PREFIX-TUNING, which only tumes soft prompts, achieve comparable performance but needs much fewer resource than FINETUNE. These results are consistent with pre- vious studies (Li and Liang, 2021; He et al., 2021) and show the effectiveness of PEFT techniques.
Comparisons
A Ablation Study We conduct ablation study to show the contributions of different techniques used in the proposed FEDSP. We separately remove the effect of using knowledge distillation (w/o KD), cross-layer sharing (w/o CS), and alternative train- ing (w/o AT). The experimental results are reported
When applying FL for privacy protection, can see that FRDPROMPT achieves similar perfor- mances with PREFIX-TUNING on all the down- stream datasets, demonstrating the effectiveness of

---

Table 3: Model performance w.r.t. different selected layers of the auxiliary models.
ARC-C
ARC-E
HellaSwag
OpenBookQA
PIQA
RACE
SciQ
GPT2-XLaor
26.5
61.2
40.9
24.2
71.0
35.2
92.8
GPT2-XLMID
22.7
60.9
39.6
15.0
70.4
35.3
90.9
GPT2-XLTOP
25.2
59.4
39.2
9.6
70.8
35.6
86.3
OPT-1.3BBor
26.8
58.0
42.1
26.0
71.9
36.6
89.9
OPT-1.3BMID
27.0
56.4
40.8
23.2
70.8
35.8
89.5
OPT 1.3-1.BBTOP
26.5
56.6
39.1
24.0
70.9
35.6
89.4
at the bottom of Table 1 and
GPT2-XL
OPT-1.3B
From these results, we can observe that remov- ing any one of the adopted techniques brings sig- nificant performance drop most of the adopted datasets. For example, when removing KD/CS/AT on the ARC-C dataset, the model performance is decreased by 8.7%/5.5%/0.9% for GPT2-XL and 4.7% for OPT-1.3B, respectively. These results show that the techniques are all neces- sary in FEDSP and have positive effects for further improving overall model performance.
72.7
Accuracy (%)
72.1
71.5
70.9
Layer Numbers
Figure Model performance w.r.t. different layer num- bers of the auxiliary models PIQA dataset.
4.5 Further Discussions
this section, we provide further discussions to better understand the proposed FRDSP.
layer for learning low-level representations, and might change slighter than other layers during the finetuning process.
Auxiliary Model The auxiliary models are first produced by the server according to the global model, and then updated by the clients based on their local data independently. In this part, com- pare different mechanisms for producing the aux- iliary model, including varying the selected layer from the global model and the layer numbers. The specific layer of bottom/midddle/tpp is 1/24/48 and 1/12/24 for GPT2-XL and OPT-1.3B, respectively As shown in Table 3, the server can select one of the layers from the bottom (denoted as BOT), the middle (denoted MID), or the top (denoted as TOP) of the global model (i.e. LLMs) produce the auxiliary model. We can ob- serve from the table that the BOT selection strat- egy has the best performances on most datasets, and the other two strategies have closed perfor- mances. For example, compared with GPT2- XLpor/OPPT-133B0or on the ARC-E dataset, the performances of GPT2-XLLOop/OPT have a degradation of 0.3% and 1.8% respectively. These results imply the fact that the bottom layer is more suitable for achieving good model alignment, as it is directly connected with the word embedding
Besides, we vary the layer numbers of the aux- iliary models, from 1 to 4, and report the experi- mental results in Figure 2. To apply the cross-layer sharing technique, the parameters of auxiliary mod- els are shared I./N times where L, is the total layer number of the global model and N is the layer num- bers auxiliary models. We conduct evaluations on the PIQA dataset, which show the proposed FEDSP is able to achieve better performances with the increase of selected layer number. For example, based on of FEDSP PIQA improves from 71,0/71.9 71.4/72.9 when the number of selected layers in- creases from 4. These results demonstrate that the proposed method is flexible to satisfy different requirements of computation resources from var- ious applications, achieving a better balancing privacy protection and model utility.
the performance
and
Efficiency Comparisons We compare the model sizes and communication costs of the proposed approach and baselines, as shown in Table and 5. The model size refers to the number of model parameters loaded by the clients, and the commu-

---

Table 4: Efficiency comparisons between FEDSP and baselines ARC-C with GPT2-XL.
5 Related Work
Parameter-efficient Finetuning With the in- creasingly larger scale of the pre-trained large language models (LLMs), finetuning the entire model becomes unaffordable for many individual researchers. As efficient adaptation of LLMs to new downstream tasks, Parameter-efficient Fine- tuning (PEFT) (Houlsby al., 2019; Lester et al., 2021; Li and Liang, 2021; Hu et al., 2021; Zaken et al., 2021; He et al., 2021; Khashabi et al., 2022) algorithms have emerged where only negligible proportions of parameters in the original LLMs are required to be updated. The main idea of PEFT is to add continuous soft prompts to the original model and only the parameters of the soft prompts need to be finetuned. For example, Liu al. (2021) and Lester et al. (2021) propose to add a few con- tinuous tunable parameters as soft prompts to the input word embeddings, and Li and Liang (2021) and Liu et al. (2022) suggest to prepend separate soft prompts to every model layer.
Method
Model Size
Comm. Cost
ZERO-SHOT
1.6B
FINETUNE
1.6B
1.6B
PEDPROMPT
1.6B
6.1M (0.4%)
FEDSP
111.1M (6.9%)
7.2M (0.5%)
Table 5: Efficiency comparisons between FEDSP and baselines ARC-C with OPT-1.3B.
Method
Model Size
Comm. Cost
ZERO-SHOT 1.3B
FINETUNE
1.3B
1.3B
FEDPROMPT
1.3B
3.9M (0.3%)
FEDSP
153.8M (1 (11.8%)
5.4M (0.4%)
nication cost refers to the number of parameters being exchanged between the server and clients for each training round. Here to better show such comparisons, we assume that the communication cost of FINETUNE in the context of FL is the same as the model size of LLMs.
Federated Learning in NLP In order to protect data privacy, federated Learning (FL) has attracted lot of attention from both academic and indus- trial (Konečný et al., 2016; McMahan al., 2017; Yang et al., 2019; Kairouz et al., 2021). With more and more language assistance products being ap- plied in real- world applications, FL has also in- creasingly appeared in the community of NLP to address the problem of privacy leakage, such as machine translation (Passban et al., 2022; Du et al.. 2023) and question answering (Chen al., 2021; Ait-Mlouk et al., 2022), and on (Li et al., 2021; Lin et al., 2022; Kuang et al., 2023; Cai al., 2023; Liu et al., 2023).
From the table, and communication cost of FEDSP are signifi- cantly less than FINETUNE (with degradation of 99.5%/99.66 66% and 93. 1..1%%/88.%% on GPT2-XL/OPT- 1.3B, respectively). Compared with FEDPROMPT, FEDSP has similar communication cost while sig- nificantly reducing the model size loaded by the clients (with a decrease of 93. 1%/8.22% on GPT2- XL/OPT-1.3B. respectively). These results demon- strate the efficiency of the proposed FEDSP for knowledge exchanging in FL course, since clients only need update the auxiliary model and soft prompts the local training process.
conclude the model size
Besides data privacy, related works are also fo- cused on the following challenges in the field of NLP: Data heterogeneity. For example. Chen et al. (2021) proposes a federated matching frame- work with a backbone-patch architecture to address the non-identical and independent distribution (non- IID) problem of the training data. Li et al. (2022) propose to split the model into a local part and a global part to handle the non-III issue. (ii) Task heterogeneity. For example, Dong et al. (2022a) propose an Assign--Then-Contaas framework collaborate heterogeneous NLP tasks. Dong et al. (2022b) propose a few-shot FL framework with an energy-based weighting algorithm to enhance the cross-task generalization ability. (iii) Efficient
Privacy Protection The proposed FEDSP pro- vides fundamental privacy protection (i.e., no shar- ing data directly) with the help of the federated learning framework. As the number of the model parameters of soft prompts is much smaller than that of the entire model, we believe that the level of mnivac-ppresevvig of the proposed method would be better or least at the same level. Further, pri- vacy protection can be enhanced by privacy protec- tion algorithms, such as differential privacy tech- niques (Triasteyn and Faltings, 2019; Wei al., 2020), to satisfy flexible protection requirements from different applications.

---

communication. For example, Passban et al. (2022) presents dynamic pulling FL method dynam- ically control the communication bandwidth. Du et al. (2023) presents a federated nearest neighbor framework to reduce the communication overhead. Although FL. can protect data privacy to some extent, it is incapable to cope with the situation when the server and clients need protect their model privacy. In this paper. we make the first attempt to incorporate the idea of using tunable soft prompts as messengers to meet the requirement of protecting both data privacy and model privacy.
models, a future direction can be how further improve and adjust the privacy protection strength. For example, how to protect the model architecture of the global models, or how to utilize differential privacy techniques to satisfy various real-world ap- plications with different protection requirements.
A Acknowledgements
This work was supported in part by the National Natural Science Foundation of China under Grant 61602013.
6 Conclusions
References
In this paper, we propose a novel FL training ap- proach, named FEDSP, with tunable soft prompts messengers to accomplish the knowledge de- livery among participants. Since the proposed FEDSP does not need to share the global model, it provides the necessary privacy protection for the global model when finetuning LLMs. These soft prompts are broadcast to the clients at each training round, plugged into an auxiliary model, and used to capture the knowledge from local data. Exten- sive experiments on various benchmarking datasets demonstrate the effectiveness of FEDSP, showing that using tunable soft prompts messengers in FL is able to protect both model privacy and data privacy, and achieve competitive performance with baselines and needs much less computation and communication costs.
Addi Ait-Mlouk, Sadi Alawadi, Salman Toor, and An- dreas Hellander. 2022. Fedqas: Privacy-aware ma- chine reading comprehension with federated learning. arXiv preprint 2rX1v:2202.04742.
Yonatan Bisk, Rowan Zellers, Ronan Le bras, Jianfeng Gao, and Yejin Choi. 2020. Piqa: Reasoning about physical commonsense in natural language. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger. Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler. Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems.
Limitations
Dongqi Cai, Yaozong Wu, Shangguang Wang. Felix Xi- aozhu Lin, and Mengwei Xu. 2023. Fffficient fed- erated learning for modern nlp. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, pages 1-16.
We propose to use tunable soft prompts as mes- sengers for useful knowledge exchange in FL. As for the limitations of this study, we conclude from the following three perspectives: (1) The proposed approach achieves privacy protection for both local data and the global model, but at the same time brings a slight performance drop compared to fine- tuning the LLMs directly. can be regarded as a trade-off between privacy protection and model utility. We hope that further research can achieve a better balance on such trade-off. (2) Some adopted techniques, such as cross-layer sharing, rely on the wransoormrrbbseer architecture, which is the most widely used architecture in LLMs. In the long run, it could be better to choose a general technique to construct the auxiliary model used in FEDSP. (3) Although we conduct the first attempt to provide protection for both local data and global
Daoyuan Chen, Yilun Huang. Zhijian Ma, Hesen Chen, Xuchen Pan, Cc Ge. Dawei Gao, Yuexiang Xie, Zhaoyang Liu, Jinyang Gao, Yaliang Li. Bolin Ding. and Jingren Zhou. 2023. Data Data-juicer: A one-stop data processing system for large language models. arXiv preprint 2339::2309.00000
Jiangui Chen, Ruqing Zhang. Jiafeng Guo, Yixing Fan, and Xueqi Cheng. 2021. Fedmatch: Federated learn- ing over heterogeneous question answering data. In Proceedingy of the 30th ACM International Confer- ence on Information Knowledge Managemens.
Sijie Cheng, Jingwen Wu, Yanghua Xiao, and Yang Liu. 2021. Fedgems: Federated learning of larger server models via selective knowledge fusion. arXiv preprini arXiv:21 10.11027.

---

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.03311.
Jakub Konečný, Brendan McMahan, Daniel Ramage, and Peter Richtárik. 2016. Federated optimization: Distributed machine learning for on-device intelli- genee. arXiv preprint 1610v.6600.02
Weirui Kuang, Bingchen Qian, Zitao Li, Daoyuan Chen, Dawei Gao. Xuchen Pan, Yuexiang Xie. Yaliang Li, Bolin Ding, and Jingren Zhou. 2023. Fedderatedscope-Ilm: A comprehensive package for fine-tuning large language models in federated learn- ing. arXiv preprint w:Xiv:2309.00300
Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question an- swering? try are, the ai2 reasoning challenge, arXiv preprint 1.X3v:88030055555
Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang. and Eduard Hovy. 2017. RACE: Large-scale ReAd- ing comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.
Chenhe Dong, Yuexiang Xic, Bolin Ding, Ying Shen, and Yaliang Li. 2022a. Collaborating heterogeneous natural language processing tasks via federated learn- ing. arxiv preprint arXiv:2212.05779.
Weilong Dong. Xinwei Wu, Junzhuo Li, Shuangzhi Wu, Chao Bian, and Deyi Xiong. 2022b. Fewfedweight: Few-shot federated learning framework across multi- ple alp tasks. arXiv preprint urX1v:2212.08354.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. Albert: A lite bert for self supf-suu learning of language representations. In International Confer- ence on Learning Representations.
Yichao Du, Zhirui Zhang, Bingzhe Wu, Lemao Liu, Tong Xu, and Enhong Chen. 2023. Federated nearest neighbor machine translation. The Eleventh Inter- national Conference on Learning Representations.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021,
The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference Empirical Methods in Natural Language Processing.
Junxian He, Chunting Zhou, Xuezhe Ma. Taylor Berg- Kirkpatrick, and Graham Neubig. 2021. Towards a unified view of parameter-effiiient transfer learning. arXiv preprint arXiv:2110.04366
Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-contrastive federated learning. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10713-10722.
Geoffrey Hinton, Oriol Vinyals, and Jeffrey Dean, 2015. Distilling the knowledge in a neural network. In NIPS Deep Learning and Representation Learning Workshop.
Xiang Lisa Li and Percy Liang, 2021. teeix-tuning:
Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficien transfer learning for nlp. In In- lernational Conference on Machine Learning, pages 2790-2799.
Zhengyang Li, Shijing Si, Jianzong Wang. and Jing Xiao. 2022. Federated split BERT for heterogeneous text classification. In International Joint Conference on Neural Networks.
Bill Yuchen Lin, Chaoyang He, Zihang 7e, Hulin Wang, Yufen Hua, Christophe Dupuy. Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, and Salman Avestimehr. 2022. FedNLP: Benchmarking federated learning methods for natural language processing tasks. In Findings of the Association for Computational Lin- guistics: NAACL 2022, pages 157-175.
Edward Hu, Yelong Shen, Phillip Wallis. Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adap- tation of large language models. arXiv preprint 1rXiv:2106.09688
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cor- mode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and Trends@ in Machine Learning.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. 2022. P-tuning: Prompt tuning can be comparable to tinettuning across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.
Daniel Khashabi, Xinxi Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Ha- jishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, and Yejin Choi. 2022. Prompt wayward- ness: The curious case of discretized interpretation of continuous prompts. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3631 1631-3
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding. Yujie Qian, Zhilin Yang. and Jie Tang. 2021. Gpt understands, too. arXiv:2103.1033
Yi Liu, Xiaohan Bi, Lei Li, Sishuo Chen, Wenkai Yang, and Xu Sun. 2023. Communication efficient feder- ated learning for multilingual neural machine transla- tion with adapter. In Findings of the Association for

---

Computational Linguisties: ACL 2023, pages 5315- 5328.
Johannes Welbl, Nelson F. Liu, and Matt Gardner. 2017. Crowdsoureing multiple choice science questions. arXiv preprint arXiv: 10t1:/707.000 77077762099
Nya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Confer- ence on Learning Representations.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Remi Louf, Morgan Funtow- icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite. Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Trans- formers: State-of-the-an natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations.
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficii learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics.
Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can suit of armor conduct elec- urcity? a new dataset for open book question an- swering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
Guangxuan Xiao, Ji Lin, and Song Han. 2023. Offsite- tuning: Transfer learning without full model. arXiv preprint arXiv:2302.0444 04200477.
Yuexiang Xie, Zhen Wang, Dawei Gao. Daoyuan Chen, Liuyi Yao, Weirui Kuang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2023. Federatedscope: A flexible federated learning platform for heterogeneity. Pro- ceedings of the VLDB Endowment, 16(5):1059-1077
OpenAI. 2023, Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
Peyman Passban, Tanya Roosta, Rahul Gupta, Ankit Chadha, and Clement Chung. 2022. Training mixed- domain translation models via federated learning. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computa- tional Linguistics.
Qiang Yang. Yang Liu, Tianjian Chen, and Yongxin Tong. 2019. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Elad Ben Zaken, Shauli Ravfogel, and Yoav Gold- berg. 2021. Bitfit: Simple parameter-efficient fine-tuning for transformer-basee masked language- models. arXiv preprini arXiv:2106.100 10799..
Teven Le Scao, Angela Fan, Christopher Akiki, EI- lie Pavlick, Suzana Ilié, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b- parameter open-access multilingual language model. arXiv preprint arXiv:2211.055 100.
Rowan Zellers. Ari Holtzman, Yonatan Bisk, Ali
Farhadi, and Yejin Choi. 2019. Hellaswag: Can machine really finish your sentence? arXiv preprint 19X50:9905.07830
Aohan Zeng. Xiao Liu, Zhengxiao Du, Zihan Wang. Hanyu Lai, Ming Ding, Zhuayi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022. Glm-130bs An open bilingual pre- pre-trained model. arXiv preprint 20..v:2210.02414..
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13977.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt. Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models. arXiv preprint arXiv:2205.0106%
Aleksei Triasteyn and Boi Faltings. 2019. Federated learning with hayesian differential privacy. In 2019 IEFE International Conference on Big Data (Big Data), pages 2587-2599
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems.
Haodong Zhao, Wei Du Fangqi Li, Peixuan Li, and Gongshen Liu. 2022. Fedprompt: Communication- efficient and privacy preserving prompt tuning in federated learning. arXiv preprint arXiv:2208. 12268.
Kang Wei, Jun Li, Ming Ding, Chuan Ma. Howard H Yang. Farhad Farokhi, Shi Jin, Tony QS Quek, and H Vincent Poor. 2020. Federated learning with differ- ential privacy: Algorithms and performance analysis. IEEE Transactions on Information Forensics and Se- curity, 15:3454-3469.
Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang. Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang. Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303. 18223.


Reviews:
[{"review_id": "MUPtNzAZKs", "reviewer": "Reviewer_4Hpf", "summary": "", "questions": "(1) Please discuss the relation between this work and\n \nCai, Dongqi, et al. \"FedAdapter: Efficient Federated Learning for Modern NLP.\" arXiv preprint arXiv:2205.10162 (2022).\nLiu, Yi, et al. \"Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter.\" arXiv preprint arXiv:2305.12449 (2023).\n\n(2) Is it possible the soft prompt in the developed federated learning framework may also leak some important information, when some approaches are used to recover the information from the soft prompts? \n\nKhashabi, Daniel, et al. \"Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.\n\n(3) In terms of performance, how does the proposed approach perform, when being compared to FedAvg, FedProx and FedOPT mentioned in\n\nLin, Bill Yuchen, et al. \"Fednlp: Benchmarking federated learning methods for natural language processing tasks.\" arXiv preprint arXiv:2104.08815 (2021).", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed comments and helpful suggestions! We make the following responses point by point to address your comments. We use **R** to denote reasons to reject and **Q** to denote questions for the authors.\n\n**R1: \"The baselines related to federated learning seem to be limited. Another type of work, where adapter-based approaches are also developed in federated learning, can be discussed or compared. Some classic federated learning approaches can also be used.\"**  \n**& Q1: \"Please discuss the relation between this work and several previous studies.\"**\n\nThank you for the comments! We are afraid that there exists a misunderstanding on the proposed method, since the discussed scenarios in this paper are significantly different from previous studies. For more details: \n\n- The mentioned related works are proposed to improve the communication efficiency in FL via applying parameter-efficient finetune techniques (such as adapters). We believe that these works make remarkable progress when the backbone model (i.e., the models that are frozen and maintained in both server and clients) are not secret, e.g., some open-source LLMs. However, when the backbone model could not be shared in FL due to some commercial or privacy concerns, these related works could not be applied.\n- What we propose in this paper is the feasible solution when the backbone model cannot be shared. We use an auxiliary model and soft prompts for local training and knowledge communication. \n- Note that similar settings (i.e., secret backbone model) are also discussed in previous studies [1] in the context of transfer learning, and such settings are reasonable as there exist some closed-source LLMs. We believe that this work is an important forward based on previous studies to handle the practical issues in some applications that own secret models such as healthcare.\n\nWe will add more details in the latest version to avoid such misunderstandings, and hope the above discussion can address your concerns. Thank you again!\n\n\n**R2: \"It is still unclear whether it is safe to pass the soft prompt (in natural, they are also model parameters) information.\"**  \n**& Q2: \"Is it possible the soft prompt in the developed federated learning framework may also leak some important information, when some approaches are used to recover the information from the soft prompts?\"**\n\nThanks a lot for your comments!\n\n- The proposed method provides fundamental privacy protection (no sharing of data directly) with the help of the federated learning framework. Meanwhile, the number of the model parameters of soft prompts is much smaller than that of the entire model, thus we believe that the level of privacy-preserving of the proposed method would be better or at least at the same level.\n- Besides, the proposed method is more robust than previous studies, since most of the backbone model would be shared in the proposed method (only one layer is shared after preprocessing) while previous studies are applied to open-source backbones.\n- Meanwhile, privacy protection can be enhanced by privacy protection algorithms (such as differential privacy, homomorphic encryption, etc.) to satisfy flexible protection requirements from different applications. Users can balance utility-privacy-cost according. \n- These privacy protection algorithms can be regarded as plugins and are orthogonal to the proposed framework, thus in this submission, we pay more attention to describing how to use the soft prompts to share knowledge without sharing the backbones.\n\nWe will add the above discussions in the latest revision accordingly. Hope the above discussion can address your concerns, thank you again!\n\n\n**Q3: \"In terms of performance, how does the proposed approach perform, when being compared to FedAvg, FedProx and FedOPT.\"**\n\nThanks a lot for your comments! We have compared the proposed method with some closed related settings such as PREFIX-TUNING (training soft prompts centralized) and FEDPROMPT (training soft prompts with FedAvg). We agree that it is a good idea to apply different federated optimization algorithms (such as FedProx and FedOPT) in both baseline and the proposed method (in the submission, the proposed method adopts FedAvg for a fair comparison) for comparison. As it is an orthogonal direction to what we focus on in this submission, we leave it to future work.\n\nWe will add the above discussions in the latest revision according to your suggestions! Thank you again!\n\n\n**For \"missing references\"**\n\nWe really appreciate your suggestions on these related works. We will add them accordingly, together with discussions to highlight the differences and advances of the proposed methods. Thank you very much for helping us to make this submission better.\n\nThank you again for the comments! We believe this submission has been further improved according to your helpful suggestions, and hope that these responses can address all your concerns and convince you to lean more toward acceptance of the paper.\n\n\nRefs:  \n[1] Xiao et al. Offsite-Tuning: Transfer Learning without Full Model. arXiv preprint arXiv:2302.04870.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) The paper studies an emerging and important problem.\n\n(2) The proposed solution has good intuitions and is easy to understand.\n\n(3) The paper is well written and presented.", "reasons_to_reject": "(1) The baselines related to federated learning seems to be limited. Another type of works, where adapter-based approaches are also developed in the federated learning, can be discussed or compared. Some classic federated learning approaches can also be used for NLP tasks and compared to the federated learning framework proposed in the paper. \n\nCai, Dongqi, et al. \"FedAdapter: Efficient Federated Learning for Modern NLP.\" arXiv preprint arXiv:2205.10162 (2022).\nLiu, Yi, et al. \"Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter.\" arXiv preprint arXiv:2305.12449 (2023).\nLin, Bill Yuchen, et al. \"Fednlp: Benchmarking federated learning methods for natural language processing tasks.\" arXiv preprint arXiv:2104.08815 (2021).\nLi, Qinbin, Bingsheng He, and Dawn Song. \"Model-contrastive federated learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\nYoon, Tehrim, et al. \"Fedmix: Approximation of mixup under mean augmented federated learning.\" arXiv preprint arXiv:2107.00233 (2021).\n\n(2) Although it is claimed in the paper that ‘FEDSP does not need to share the global model, it provides the necessary privacy protection for the global model when finetuning LLMs.’, it is still unclear whether it is safe to pass the soft prompt (in natural, they are also model parameters) information. There are some works which try to recover discrete tokens from soft prompt parameters. It is unclear whether the soft prompt in the developed federated learning framework may also leak some important information, when some approaches are used to recover the information from the soft prompts. It seems that limited discussion are included in the paper.\n\nKhashabi, Daniel, et al. \"Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.", "questions_to_authors": "(1) Please discuss the relation between this work and\n \nCai, Dongqi, et al. \"FedAdapter: Efficient Federated Learning for Modern NLP.\" arXiv preprint arXiv:2205.10162 (2022).\nLiu, Yi, et al. \"Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter.\" arXiv preprint arXiv:2305.12449 (2023).\n\n(2) Is it possible the soft prompt in the developed federated learning framework may also leak some important information, when some approaches are used to recover the information from the soft prompts? \n\nKhashabi, Daniel, et al. \"Prompt Waywardness: The Curious Case of Discretized Interpretation of Continuous Prompts.\" Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.\n\n(3) In terms of performance, how does the proposed approach perform, when being compared to FedAvg, FedProx and FedOPT mentioned in\n\nLin, Bill Yuchen, et al. \"Fednlp: Benchmarking federated learning methods for natural language processing tasks.\" arXiv preprint arXiv:2104.08815 (2021).", "paper_topic_and_main_contributions": "Federated learning (FL) enables multiple participants to collaboratively train machine learning models from decentralized data sources, alleviating privacy concerns raised by directly sharing local data. However, the lack of protection for model privacy in FL becomes an unneglectable challenge, especially when people want to federally finetune models based on a private large language model. This paper proposes a novel FL training approach that accomplishes information exchange among participants via tunable soft prompts. These soft prompts are updated and transmitted between the server and clients, taking over the duty of the global model parameters and serving as messengers to deliver useful knowledge in the local data and global model.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Cai, Dongqi, et al. \"FedAdapter: Efficient Federated Learning for Modern NLP.\" arXiv preprint arXiv:2205.10162 (2022).\nLiu, Yi, et al. \"Communication Efficient Federated Learning for Multilingual Neural Machine Translation with Adapter.\" arXiv preprint arXiv:2305.12449 (2023).\nLin, Bill Yuchen, et al. \"Fednlp: Benchmarking federated learning methods for natural language processing tasks.\" arXiv preprint arXiv:2104.08815 (2021).\nLi, Qinbin, Bingsheng He, and Dawn Song. \"Model-contrastive federated learning.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021.\nYoon, Tehrim, et al. \"Fedmix: Approximation of mixup under mean augmented federated learning.\" arXiv preprint arXiv:2107.00233 (2021)."}}, {"review_id": "dWsp7rmkY6", "reviewer": "Reviewer_Mx16", "summary": "", "questions": "Please address the concern in the Reasons To Reject section.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed comments and helpful suggestions! We make the following responses point by point to address your comments. We use **R** to denote reasons to reject and **Q** to denote questions for the authors.\n\n**R: \"The only concern is about the construction of the auxiliary model.\"**\n\nThank you for the comments. \n\n- Compared to other model parameters, the word embeddings have much fewer parameters. Thus we believe that the level of privacy-preserving of the proposed method would be better (or at least no worse than) previous studies that propose to share the whole model parameters or gradients.\n- It is a very nice idea to adopt other LLMs for better privacy preservation. In fact, there might exist a trade-off between model utility and model privacy here, since using the same LLM might achieve a better model performance as the relationships between server's model and client's model, while using a different LLM can be a better choice for privacy protection. We will add more discussion in the latest version and consider these experiments as future work.\n\nThank you again for the comments! We believe this submission has been further improved according to your helpful suggestions, and hope that these responses can address all your concerns and convince you to lean more toward acceptance of the paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Protecting the privacy of large models in FL is an interesting research problem worth studying.\n2. The paper is well-written, and the methods are clearly presented.\n3. The authors have validated the effectiveness of FedSP on seven datasets and promised to release the source code in the future.", "reasons_to_reject": "The paper does not have many issues. The only concern is about the construction of the auxiliary model. The author used a part of the global model to construct the auxiliary model, which I suspect may also include the word embedding layer that already contains a significant amount of information from the original LLM. A more appropriate approach would be to use a smaller, open-source LLM that is completely unrelated to the global model for experimentation.", "questions_to_authors": "Please address the concern in the Reasons To Reject section.", "paper_topic_and_main_contributions": "The paper proposes a novel federated learning (FL) approach, named FedSP, that uses tunable soft prompts as messengers to exchange knowledge among participants without sharing the global model. FedSP involves initializing an auxiliary model for each client, sending up-to-date soft prompts to selected clients, and performing an alternative local training process for global model alignment and local knowledge capturing. The paper conducts experiments with two LLMs (GPT2-XL and OPT-1.3B) on seven question-answering datasets and shows that the proposed approach achieves competitive performance with baselines while reducing model size and communication cost.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "w1PFdpvqXe", "reviewer": "Reviewer_d21D", "summary": "", "questions": "1. What is the meaning of the $auxiliary\\ model$? What is the definition of this term? If this term is from a paper, a reference is needed. Or if it is created by the author, a definition or some examples at least should be given. Is the auxiliary model a part of LLM? Is the local model the same as the auxiliary model? Is the local model's structure the same as the global model? In the standard FL, the global model should be the same as the local model for weight aggregation and updating. For model privacy, the global model cannot share its weight with the clients. I am puzzled about whether the global model is the same as the local model.\n2. Why is there centralized training in the baselines? If this paper is about FL, centralized training is very likely better than FL. \n3. What is the model structure of the global model and the client model in your experiments?\n4. If there is no knowledge distillation (KD), how can you initialize the local model since you want to protect the model privacy in Table 1 and Table 2?\n5. If no alternative training (AT) exists, how can you train the local model in Table 1 and Table 2?\n6. What is the specific layer of bottom, middle, and top in Table 3?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your detailed comments and helpful suggestions! We make the following responses point by point to address your comments. We use **R** to denote reasons to reject and **Q** to denote questions for the authors.\n\n**R1: \"There is no contribution list at the end of Introduction part.\"**\n\nThank you for the comments! The contributions of this paper are summarized as:\n\n- In this study, what we focus on is a feasible solution when the global model in federated learning cannot be shared due to some commercial or privacy concerns. Such a scenario is reasonable and practical, but could not be handled by previous studies.\n- The proposed method, called FedSP, is able to accomplish information exchange among participants via tunable soft prompts. These soft prompts are updated and transmitted between the server and clients, taking over the duty of the global model parameters and serving as messengers. FedSP makes it possible and meaningful to apply federal training when the global model cannot be shared (e.g., closed-source LLMs). \n- We conduct a series of experiments with two LLMs (GPT2-XL and OPT-1.3B), showing that FedSP achieves competitive performance compared to baseline methods. We believe that this work is an important forward based on previous studies to handle the practical issues in some applications that own secret models such as healthcare.\n\nWe will add these contributions accordingly in the latest version for better understanding. Thank you again for your valuable suggestions to make our paper better!\n\n\n**R2: \"The performance of FedSP is not the best in Table 1 and Table 2 on some datasets.\"**\n\nThanks a lot for your comments! \n\n- The baseline methods include FINETUNE (centralized training entire model),  PREFIX-TUNING (centralized training soft prompts) and FEDPROMPT (training soft prompts with FedAvg). These methods actually have a more relaxed assumption on the privacy concerns (e.g., sharing the data or global model) compared to FedSP (data and global model could not be shared), so it is not surprising that FedSP is not the best in the reported results.\n- As we all know that there exists a trade-off between model utility and privacy protection, it is still important and meaningful to compare FedSP with these baselines to show how the proposed method balances such trade-off. The experiments show that FedSP achieves competitive performance compared to baseline methods, which implies that FedSP is an effective solution to satisfy a more strict privacy assumption with acceptable performance changes.\n\nWe will add the above discussion in the latest version to make them more clear, thank you again!\n\n\n**R3: \"The author should do more experiments about FL settings, like the number of clients, communication rounds.\"**\n\nThanks a lot for your helpful suggestions! We will report more experimental results in the latest version, thanks again.\n\n\n**R4: \"The settings of the global and client model are not clear, like the model structure.\"**\n\nThanks a lot for your comments! The global model is the transformer-based language model (e.g., GPT2-XL or OPT-1.3B), which contains an embedding layer, several similar transformer blocks, and an output layer (please refer to section 4.1 or [1, 2] for more details). For the client model, it is similar to the global model but only contains one layer. Both the global model and client model are added with some soft prompts following [3], where continuous prompts are applied for every layer input of the language model.\n\nWe will add these details to make them more clearly in the latest version, thank you again.\n\n\n**Q1: \"What is the meaning of the auxiliary model? What is the definition of this term?\"**\n\nThanks a lot for your comments! The auxiliary model is used for knowledge communication, together with soft prompts, which is one corresponding layer of the global model. We use the \"auxiliary model\" to distinguish it from the global model (i.e., the entire model). \nWe will add these details to make them more clearly in the latest version, thank you again!\n\n\n**Q2: \"Why is there centralized training in the baselines?\"**\n\nThanks a lot for your comments! \n\n- We add centralized training in the baselines to show that FedSP achieves a good balance between privacy and utility. In other words, the centralized training methods can be regarded as the oracle to FL methods. Such comparisons are also used in previous studies.\n- For more details about the baselines, please refer to R2: \"The performance of FedSP is not the best in Table 1 and Table 2 on some datasets.\".\n\nWe will add the above discussion in the latest version to make them more clear, thank you again!\n\n\n**Q3: \"What is the model structure of the global model and the client model in your experiments?\"**\n\nThanks a lot for your comments! Please refer to R4: \"The settings of the global and client model are not clear, like the model structure.\"\n\n\n**Q4: \"If there is no knowledge distillation (KD), how can you initialize the local model since you want to protect the model privacy in Table 1 and Table 2?\"**\n\nThanks a lot for your comments! When there is no KD, the local model is initialized from the corresponding layers of the global model. As the layer number of the local model is much smaller than that of the entire model, and other layers would not be shared during the whole training process, we believe that it would not bring additional model privacy issues compared to previous studies that share the entire global  model. \n\nWe will add the above discussion in the latest version, thank you again!\n\n\n**Q5: \"If no alternative training (AT) exists, how can you train the local model in Table 1 and Table 2?\"**\n\nThanks a lot for your comments! When no AT exists, it implies that we only finetune the soft prompts in each local model while keeping other model parameters frozen. We will add these details to make them more clearly in the latest version, thank you again.\n\n\n**Q6: \"What is the specific layer of bottom, middle, and top in Table 3?\"**\n\nThanks a lot for your comments! The specific layer of bottom/middle/top is 1/24/48 and 1/12/24 for the LLM of GPT2-XL and OPT-1.3B, respectively. We will add these details to make them more clearly in the latest version, thank you again.\n\nWe believe this submission has been further improved according to your helpful suggestions, and wish the above responses can answer all your questions and address all your concerns! Hope these responses can convince you to lean more toward acceptance of the paper, thank you again for the detailed comments! \n\n\nRefs:  \n[1] Radford et al. Language Models are Unsupervised Multitask Learners. 2019.  \n[2] Zhang et al. OPT: Open Pre-trained Transformer Language Models. arXiv preprint arXiv:2205.01068.  \n[3] Liu et al. P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks. ACL 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea itself is interesting. Model privacy is a problem in this period if we want to use FL but do not want to share the model weight.\n2. The discussion is detailed.\n3. The code is complete for reproducing the results.\n4. The experiments are complete.", "reasons_to_reject": "1. The writing is hard to follow.  There is no contribution list at the end of Introduction part. I read it several times but I am sorry that I cannot catch your theme. Is this paper mainly about model privacy in FL (FedSP) or soft prompt usage?\n2. If the theme is mainly about FedSP, the performance of FedSP is not the best in Table 1 and Table 2 on some datasets.\n3. If the paper is about FedSP, the author should do more experiments about FL settings, like the number of clients, communication rounds, etc.\n4. The settings of the global and client model are not clear, like the model structure, etc. ", "questions_to_authors": "1. What is the meaning of the $auxiliary\\ model$? What is the definition of this term? If this term is from a paper, a reference is needed. Or if it is created by the author, a definition or some examples at least should be given. Is the auxiliary model a part of LLM? Is the local model the same as the auxiliary model? Is the local model's structure the same as the global model? In the standard FL, the global model should be the same as the local model for weight aggregation and updating. For model privacy, the global model cannot share its weight with the clients. I am puzzled about whether the global model is the same as the local model.\n2. Why is there centralized training in the baselines? If this paper is about FL, centralized training is very likely better than FL. \n3. What is the model structure of the global model and the client model in your experiments?\n4. If there is no knowledge distillation (KD), how can you initialize the local model since you want to protect the model privacy in Table 1 and Table 2?\n5. If no alternative training (AT) exists, how can you train the local model in Table 1 and Table 2?\n6. What is the specific layer of bottom, middle, and top in Table 3?", "paper_topic_and_main_contributions": "This paper introduces a topic called model privacy in this large language model (LLM) period, which is interesting and somehow necessary in academia. To protect model privacy in FL, the authors utilize knowledge distillation initialization, cross-layer parameter sharing, alternative training and parameter efficient fine-tuning. By testing on two datasets and two LLMs, the author illustrates a way of LLM training in FL with model privacy protected. ", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "The prompt finetuning in FL needs the following reference:\n\n[1] Guo, Tao, et al. \"PromptFL: Let Federated Participants Cooperatively Learn Prompts Instead of Models--Federated Learning in Age of Foundation Model.\" arXiv preprint arXiv:2208.11625 (2022).\n\n[2] Zhao, Haodong, et al. \"Reduce communication costs and preserve privacy: Prompt tuning method in federated learning.\" arXiv preprint arXiv:2208.12268 (2022)."}}]

Author rebuttal:

Meta-review: The reviewers appreciated that the paper tackles an interesting and important problem. The paper also extensively evaluates their proposed approach. However, there were several concerns about how the work compares to different baselines (some of which the authors rebut are not applicable, esp. when the backbone model is private). 

This is definitely an interesting direction and I encourage the authors to revise and improve their draft—for instance, it could benefit from clearer and more detailed presentation (reviewers note that they find it hard to follow, and some of the contributions are not immediately clear). The paper could also discuss some related work that was missed in this iteration.|Reviewers acknowledged the paper's engagement with a significant and intriguing problem. They also commended the comprehensive evaluation of the proposed approach. Nevertheless, concerns arose regarding the comparisons with various baselines, with the authors noting that some are not applicable, particularly when the backbone model is private.

This direction is undeniably captivating, and I encourage the authors to refine and enhance their manuscript. Improvements could include a more lucid and detailed presentation to aid comprehension, addressing reviewers' difficulties in following the content and clarifying certain contributions. Additionally, the paper should consider incorporating relevant missed related work.
Decision: Accept-Findings