Personalized Federated Learning for Text Classification with Gradient-Free Prompt Tuning (2024)

Abstract: In this paper, we study personalized federated learning for text classification with Pretrained Language Models (PLMs). We identify two challenges in efficiently leveraging PLMs for personalized federated learning: 1) Communication. PLMs are usually large in size, e.g., with hundreds of millions of parameters, inducing huge communication cost in a federated setting. 2) Local Training. Training with PLMs generally requires back-propagation, during which memory consumption can be several times that of the forward-propagation. This may not be affordable when the PLMs are trained locally on the clients that are resource constrained, e.g., mobile devices with limited access to memory resources. Additionally, the proprietary PLMs can be provided as concealed APIs, for which the back-propagation operations may not be available. In solving these, we propose a training framework that includes an approach of discrete local search for gradient-free local training, along with a compression mechanism inspired from the linear word analogy that allows communicating with discretely indexed tokens, thus significantly reducing the communication cost. Experiments show that our gradient-free framework achieves superior performance compared with baselines.

## introduction personalized federated learning @xcite @xcite involves collaborative training with non-shareable private data from multiple clients. for each client, we aim to train a personalized model that fits to its local data, leveraging knowledge from other clients. personalized federated learning has been increasingly attended in the federated learning community due to its ability to account for data heterogeneity across clients @xcite . on the other hand, the advent of pretrained language models (plms) @xcite has yielded remarkable performance for natural language processing, e.g., text classification. ## related work federated learning with plms: the sheer size of the plms @xcite poses challenges to federated learning due to both high communication cost and large memory footprint during local training. previous works @xcite @xcite of federated learning with plms mostly targets the training efficiency in terms of the communication cost. for instance, @xcite propose to reduce the communication cost by only communicating the lower layers of the plms between server and clients, while the upper layers are trained only with local data. inspired by the superior performance and efficiency of prompt tuning @xcite , @xcite @xcite propose to further reduce the communication cost via only training and communicating the continuous prompt embeddings. the drawback of these works is that they all require training with gradients, neglecting the huge memory consumption caused by back-propagating through the plms. this can be problematic in federated learning for clients with constrained computation resources, e.g., edge devices that have limited memory capacity. gradient-free training with plms: sun et al. (2022b); cao et al. (2023) propose language-model-as-a-service (lmaas) that fine tunes the pretrained prompt embeddings of the plms with cma-es (hansen and ostermeier, 2001), a gradient-free method that only requires forwardpropagation. this setting requires the client data being transferred to an external server with the api of plms, thus violating the privacy-preserving principle of federated learning. deng et al. (2022); diao et al. (2022) model the prompts for the plm inputs with a prompt generator that is trained with gradients from reinforcement learning. in this way, the back-propagation is not on parameters of the plm but the prompt generator. this may not be suitable for federated learning, since the prompt generators (e.g., implemented with another plm) can introduce additional large computation cost for clients during local training. hou et al. (2022); prasad et al. (2022) also study gradient-free training of plms, but it is unclear how to apply their approach for federated learning. to illustrate, hou et al. (2022) adopts boosting with prompts, requiring ten times the computation for model inference compared to without boosting, thus is not compatible with clients equipped with constrained computation resources. importantly, none of the works above study federated learning. in this way, the only trainable parameter for client i is the prompt p i . the training loss for client i is, when training with personalized federated learning for text classification, the general objective is to find {p i } m @xmath0, while keeping {d i } m @xmath1. this is achieved via coordinating the training with a server that iteratively receives {p i } m @xmath2, denoted as p in section 4.1. unlike the plms with online apis (e.g. gpt-3.5/4 (openai, 2023)) that requires uploading user data to an external server, it is reasonable that the plms is deployed locally (i.e., without data uploading), for better data privacy with federated learning. ## conclusions in this paper, we propose a gradient-free framework that trains with discrete local search on natural language token during personalized federated learning. compared with gradient-based approaches, the discrete local search circumvents gradient computation and saves the huge memory consumption caused by back-propagation. we additionally propose a compression mechanism inspired by linear word analogy that allows the server-client communication with discretely indexed tokens. experiments show that our framework can achieve superior performance compared to baselines.