FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models

## Abstract

Abstract In recent years, large language models (LLMs) have significantly advanced the field of natural language processing (NLP). By fine-tuning LLMs with data from specific scenarios, these foundation models can better adapt to various downstream tasks. However, the fine-tuning process poses privacy leakage risks, particularly in centralized data processing scenarios. To address user privacy concerns, federated learning (FL) has been introduced to mitigate the risks associated with centralized data collection from multiple sources. Nevertheless, the privacy of LLMs themselves is equally critical, as potential malicious attacks challenge their security, an issue that has received limited attention in current research. Consequently, establishing a trusted multi-party model fine-tuning environment is essential. Additionally, the local deployment of large LLMs incurs significant storage costs and high computational demands. To address these challenges, we propose for the first time a federated discrete and transferable prompt tuning, namely FedDTPT, for black-box large language models. In the client optimization phase, we adopt a token-level discrete prompt optimization method that leverages a feedback loop based on prediction accuracy to drive gradient-free prompt optimization through the MLM API. For server optimization, we employ an attention mechanism based on semantic similarity to filter all local prompt tokens, along with an embedding distance elbow detection and DBSCAN clustering strategy to enhance the filtering process. Experimental results demonstrate that, compared to state-of-the-art methods, our approach achieves higher accuracy, reduced communication overhead, and robustness to non-iid data in a black-box setting. Moreover, the optimized prompts are transferable.

## 1 Introducation

Large language models (LLMs) have demonstrated significant success across numerous natural language processing (NLP) tasks (Brown et al., ; Devlin et al., ; Radford et al., ). Typically, these models are trained on a vast text corpus and then applied to various downstream tasks through fine-tuning or prompt tuning. However, task-specific data is often necessary for tuning pre-trained LLMs, and this process typically relies on user-labeled data. In practice, securely leveraging these labeled data presents challenges. Data must be collected and stored for training purposes, but sharing and exchanging sensitive information can pose serious security risks and raise privacy concerns. To mitigate the risk of potential data leakage, federated learning (FL) is proposed. FL enables multiple devices to collaboratively fine-tune pre-trained LLMs on decentralized data while maintaining data privacy. Recent work, such as the bilevel optimization method (Li et al., ), has demonstrated efficient strategies to reduce communication overhead and improve optimization performance in FL scenarios. Additionally, federated object detection frameworks (Kim et al., ) and federated conditional stochastic optimization (Wu et al., ) have provided further insights into addressing communication and computational challenges in decentralized learning. Privacy and security remain critical in FL settings, and proactive defenses against model poisoning attacks, such as RECESS (Yan et al., ), help safeguard model integrity while fine-tuning LLMs in federated environments. Moreover, techniques like personalized federated learning (Yan et al., ) have introduced new ways to enhance the adaptability of global models to specific client data, addressing the heterogeneity often encountered in FL systems.

When applying federated learning (FL) for tuning pre-trained LLMs, existing approaches can be categorized into federated fine-tuning and federated prompt tuning. However, both methods have their limitations. For federated fine-tuning, the primary challenges include: (1) clients’ limited access to the parameters of pre-trained language models (PLMs), (2) significant computational and storage demands on local clients, and (3) high communication overhead within the FL system. These factors make federated fine-tuning impractical in real-world scenarios. In practice, devices primarily interact with LLMs by invoking LLM APIs, which do not grant clients access to model parameters, thus preventing local training. Moreover, even if access were available, devices with limited computational resources would struggle to perform local LLM fine-tuning (Zhou et al., ). Several approaches have been proposed to address the challenges posed by client heterogeneity and communication costs, such as leveraging model architectures designed to improve performance in FL systems despite data heterogeneity (Pieri et al., ), as well as bilevel optimization methods that offer communication-efficient solutions for FL systems (Yang et al., ). Additionally, methods like dynamic personalized federated learning (Panchal et al., ), model reassembly techniques (Wang et al., ), and federated multi-objective optimization frameworks (Yang et al., ) offer solutions for efficient model adaptation in decentralized environments. These innovations, which target the optimization of client-specific models and data distribution challenges, may also inform strategies for fine-tuning models in decentralized contexts.

An alternative approach, federated prompt tuning, as proposed by FedBPT (Sun et al., ), focuses on optimizing continuous prompts injected into text while keeping the PLM parameters frozen. Although this method reduces computational costs for clients, continuous prompts still face several limitations: (1) they are model-specific and cannot be directly applied to prediction APIs, which only accept discrete inputs, (2) continuous prompts lack interpretability, and (3) they lack transferability, meaning they cannot be seamlessly applied to other LLMs. To improve communication efficiency, methods like spectral co-distillation (Chen et al., ) and one-pass distribution sketches (Liu et al., ) have been explored, targeting efficient aggregation and reduced overhead. Furthermore, the issue of communication efficiency and local model performance trade-offs has been explored in works (Li & Huang, ), where the tension between local client computations and global model performance is thoroughly examined, providing further insight into optimizing federated learning strategies.

To address the aforementioned challenges, we propose FedDTPT, On the client side, we employ a token-level discrete prompt tuning strategy. Given the absence of a probability distribution in the inference results, we implement gradient-free prompts optimization through a feedback loop based on prediction accuracy. On the server side, we utilize an attention mechanism grounded in semantic similarity to filter prompt tokens from all clients. This mechanism identifies the most representative discrete tokens. Additionally, we enhance the filtering effectiveness by employing an inflection point detection in embedding distances and a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) clustering strategy. We conducted experiments on multiple datasets using SOTA PLMs. The results indicate that, in comparison to the current state-of-the-art techniques, our methodology attains superior accuracy, diminished communication expenses, and resilience to non-iid data within a black-box framework. Furthermore, the refined prompts exhibit transferability. Our contributions include:

- •
Problem Novelty: In this work, we introduce a new problem setting: discrete prompt learning in black-box federated learning. This setting enables the learning of transferable and interpretable prompts while safeguarding both the privacy of the server’s model parameters and the client’s data.
- •
Approach Novelty: In this work, we propose FedDTPT, a novel discrete prompt learning framework in black-box federated learning scenarios. FedDTPT utilizes the novel token-level optimization strategy to update the client prompt and a token selection method based on semantic similarity to aggregate the discrete prompt.
- •
Experimental effect: Our method achieves high accuracy and low communication overhead, and its optimized prompts exhibit transferability.

## 2 Background & Related Work

LLMs as API Service. Due to the significant computational demands of large language models (LLMs), an increasing number of LLMs are being deployed on servers as API services. From the model supplier’s perspective, this approach allows them to retain proprietary control over their models, avoiding open sourcing due to commercial considerations and the risk of misuse. From the user’s perspective, even when pre-trained LLMs are available, running them locally is often prohibitively expensive or even infeasible due to hardware constraints and the need for continuous updates (Bommasani et al., ). Given these advantages, deploying LLMs as API services has become a mainstream approach and is now the dominant trend.

Federated Learning. Federated Learning (FL) is a decentralized machine learning approach where multiple clients collaboratively train a model while keeping their data local, ensuring privacy (Konečnỳ, ). For model suppliers, FL enables large-scale training without accessing user data, reducing liability and complying with privacy regulations like GDPR (^1^11[https://gdpr-info.eu/](https://gdpr-info.eu/)). For users, it allows participation in model improvements while maintaining control over their data. Although FL offers privacy benefits, challenges like data heterogeneity, communication costs, and system differences remain key research areas. FL is increasingly applied to LLMs, especially in privacy-sensitive applications, making it a critical tool in privacy-preserving AI.

Prompt Tuning. Prompt tuning has gained considerable attention in the field of large language models (LLMs). Its goal is to search for an optimal prompt using minimal examples to guide an LLM towards generating the desired output for a specific downstream task. In NLP applications, there are two main types of prompt tuning methods: (1) continuous prompt tuning and (2) discrete prompt tuning (Liu et al., ). In continuous prompt tuning, a sequence of continuous vectors is appended to the input text embedding. Unlike discrete prompt, which operates at the vocabulary level, continuous prompt tuning (Li & Liang, ) optimizes the prompt directly in the embedding space.
In contrast, discrete prompt tuning involves a sequence of discrete tokens, which remain interpretable to humans.

## 3 Method

### 3.1 Problem Formulation

Prompt tuning is a widely adopted Parameter-Efficient Fine-Tuning (PEFT) method for large language models (LLMs). The prompts are optimized to adapt the model to specific downstream tasks. Discrete prompt tuning refers to the independent optimization of discrete tokens $p_{n}\in\mathcal{P}$ within the prompt set $\mathcal{P}$, where $n$ denotes the number of tokens in $\mathcal{P}$. This approach is more interpretable than continuous prompt tuning strategies, such as soft prompt tuning. In a federated learning context, federated discrete prompt tuning involves each client $k$, where $k\in\mathcal{K}$, transmitting their local prompts $\mathcal{P}_{k}=\{\mathbf{p}_{k}^{n}\}_{n=1}^{N}$ to a central server for a knowledge exchange based on discrete prompts. The aggregated global prompt $\mathcal{P}_{F}=\{\mathbf{p}_{F}^{n}\}_{n=1}^{N}$is then distributed back to all clients, where it is further fine-tuned on $\mathcal{D}_{k}=\{(\mathbf{x}_{k},\mathbf{y}_{k})\}_{k=1}^{K}$ be a private local dataset in the $k$-th client for personalized adaptation. The objective in this federated scenario can be expressed as:

$$ $P_{k}^{*}=\arg\min_{P_{F}}\sum_{k=1}^{K}w_{k}L_{k}\left(f\left(P_{F};D_{k} \right)\right),$ (1) $$

where $n$ is the number of tokens in $\mathcal{P}$, and $K$ represents the number of clients involved. Prompt tuning based on black-box LLMs refers to the process where the large model’s parameters are entirely fixed, and the prompts are treated as learnable parameters. Since the gradients of the LLM are inaccessible, gradient-free zeroth-order optimization methods are commonly used instead of traditional backpropagation techniques. Compared to standard prompt tuning, pure black-box prompt tuning is a more challenging optimization task. Since the inference result of the LLM prediction API, represented as $f(\mathcal{P};X)$, is purely textual and does not provide a probability distribution, Eq. (2), which relies on one-hot labels, is no longer applicable. Consequently, prompt optimization is performed solely at the token level, and we accordingly use a more direct measure of accuracy as the optimization objective:

$$ $P_{k}^{*}=\arg\max_{P_{F}}\sum_{k=1}^{K}w_{k}A_{k}\left(f\left(P_{F};D_{k} \right)\right),$ (2) $$

where $\textit{A}_{k}$ is the accuracy in client k.

### 3.2 Design Overview

The overview of FedDTPT as shown in Figure 1. In the client optimization phase of FedDTPT, we adopt a token-level discrete prompt tuning strategy that establishes a new feedback mechanism for inference results to enable gradient-free prompt optimization. During the federated learning stage, we employ a semantic similarity-based attention mechanism to sample prompt tokens from all clients, selecting the most representative discrete tokens to construct optimized prompts. This approach effectively facilitates knowledge transfer across clients while preserving privacy. At the beginning of the optimization process, a public dataset $D_{g}$, containing representative samples, is deployed to each client to assist in computing the prediction accuracy during local prompt tuning. In each global communication round, the server first broadcasts a global prompt to all clients. In the initial round, this prompt is based on the global task and can either be carefully designed or straightforward. Subsequently, each client $k$ uses the MLM API to fine-tune the global prompt, recording the tuning information. The tuned prompt is then input into the LLM prediction API to obtain inference results and calculate accuracy. The accuracy and tuning information are aggregated as optimization feedback and fed back to the MLM API for further fine-tuning. Upon completion of local optimization, all clients upload their local prompts to the server for knowledge aggregation. The server maps the discrete tokens of all prompts to a high-dimensional latent space and employs a clustering strategy based on the secondary-range elbow judgement strategy and DBSCAN approach to cluster the embeddings. Finally, a latent space similarity-based attention mechanism is applied to sample the embeddings and generate a global prompt.

Figure: Figure 1: The structure of FedDTPT. The client uses prediction results as feedback to drive the MLM API for discrete prompt optimization. The locally optimized prompts are then uploaded to the server, where tokens are mapped to a high-dimensional latent space. Similarity calculations on these high-dimensional embeddings yield weight values $W$, and a clustering strategy is applied to select high-weight tokens. These tokens are then combined to form a global prompt, which is subsequently distributed back to the clients.
Refer to caption: x1.png

### 3.3 Client Prompt Instruction Tuning

Unlike existing black-box prompt tuning tasks, in a purely black-box setting, large language models only output prediction text without probability distributions. The absence of loss information necessitates that prompt optimization be performed solely at the token level, posing significant challenges. In the client-side optimization phase, we set accuracy improvement as the primary objective and leverage the contextual understanding capabilities of masked language models (MLMs) to achieve prompt tuning. Furthermore, we establish an inference feedback loop, which, compared to random prompt optimization using MLMs, creates a closed loop between forward inference and result feedback. This approach allows the MLM to make informed predictions based on comprehensive historical information.

Specifically, the client first receives the global prompt dispatched by the server, uses the MLM for tuning, and stores the modification details.
The optimized prompt is then combined with input $x_{k}$ and fed into the LLM Inference API for prediction. By comparing the inference results with the labels $y_{k}$, we calculate the accuracy on a batch basis.
Finally, in subsequent iterations, the MLM receives both the accumulated tuning modifications and accuracy results along with the prompt to be optimized. This iterative process allows the MLM to perform more informed and effective tuning. The optimization process is detailed in Algorithm 1.

Figure: Algorithm 1 Token-level Prompt Optimization with Inference Feedback for Client $k$

Additionally, to address potential data imbalance during accuracy calculation in each iteration, we introduce a small, balanced public dataset to assist in accuracy computation. Specifically, during the accuracy calculation for each batch of client $k$’s data, the public dataset is incorporated as auxiliary data. This approach effectively mitigates the impact of data imbalance and helps to counteract non-iid data distribution issues.

### 3.4 Server Prompt Instruction Aggregation

During client-side optimization, each client sends its locally optimized prompt to the server for knowledge exchange. Since clients can only access token-level information, traditional global aggregation strategies, such as simple weighted averaging, are difficult to implement. To address this, we propose an attention mechanism based on semantic similarity, combined with high-dimensional clustering methods, to effectively select and merge important tokens, thereby generating a globally optimized prompt. The detailed methodology is outlined as follows:

Mapping Tokens to a High-Dimensional Latent Space. Each token from the prompts generated by the clients is mapped to a high-dimensional latent space. Given the need for robust contextual understanding, leveraging the embedding layers of pre-trained language models (MLMs) like BERT or RoBERTa is well-suited for this purpose, as they can project semantically similar tokens to proximate positions in the latent space. Let $\mathbf{P}_{k}=\{\mathbf{p}_{k}^{1},\mathbf{p}_{k}^{2},\ldots,\mathbf{p}_{k}^{
N}\}$ represent the sequence of discrete tokens generated by the $k$-th client, where $k\in\{1,2,\ldots,K\}$ and $N$ denotes the number of tokens in each prompt. Each token $\mathbf{p}_{k}^{n}$ is mapped to a high-dimensional embedding through a function $z$, resulting in an embedding vector $E_{k}^{n}$. The mapping function $z$ can be formally expressed as $z:\mathbf{P}_{k}\rightarrow\mathbb{R}^{N\times d},\mathbf{P}_{k}\mapsto\mathbf
{E}_{k}=\{E_{k}^{1},E_{k}^{2},\ldots,E_{k}^{N}\}$, where $E_{k}^{n}=z(\mathbf{p}_{k}^{n})\in\mathbb{R}^{d}$ is the high-dimensional embedding vector corresponding to the token $\mathbf{p}_{k}^{n}$, $d$ is the dimensionality of the latent space, and $\mathbf{E}_{k}$ is the matrix of embeddings for all tokens in the $k$-th client’s prompt. To incorporate context and semantics into the embeddings, the mapping function $z$ may depend on additional parameters, such as contextual weights $\theta$ from a pre-trained language model.

$$ $E_{k}^{n}=z(\mathbf{p}_{k}^{n};\theta)=\text{MLM}_{\theta}(\mathbf{p}_{k}^{n})$ (3) $$

where, $\theta$ represents the parameters of the pre-trained language model (MLM), such as BERT or RoBERTa, $\text{MLM}_{\theta}$ denotes the model’s embedding layer that captures the context and semantic similarity of each token. Therefore, the overall mapping process for all tokens from all clients can be expressed as a set:

$$ $\mathcal{E}=\bigcup_{k=1}^{K}\mathbf{E}_{k}=\bigcup_{k=1}^{K}\{E_{k}^{1},E_{k} ^{2},\ldots,E_{k}^{N}\}=\bigcup_{k=1}^{K}\{z(\mathbf{p}_{k}^{1};\theta),z( \mathbf{p}_{k}^{2};\theta),\ldots,z(\mathbf{p}_{k}^{N};\theta)\}$ (4) $$

where $\mathcal{E}$ represents the set of all high-dimensional embeddings for tokens across all clients.

Attention-Based Weight Calculation via Semantic Similarity. To compute the semantic similarity between tokens, we use the cosine similarity between their high-dimensional embeddings. For a token $\mathbf{p}_{k}^{n}$ from the $k$-th client and a token $\mathbf{p}_{k^{\prime}}^{n^{\prime}}$ from another prompt (client $k^{\prime}$), the cosine similarity is given by:

$$ $\text{sim}(E_{k}^{n},E_{k^{\prime}}^{n^{\prime}})=\frac{E_{k}^{n}\cdot E_{k^{ \prime}}^{n^{\prime}}}{\|E_{k}^{n}\|\|E_{k^{\prime}}^{n^{\prime}}\|}$ (5) $$

where:$E_{k}^{n}\cdot E_{k^{\prime}}^{n^{\prime}}$ denotes the dot product of the embeddings. $\|E_{k}^{n}\|$ and $\|E_{k^{\prime}}^{n^{\prime}}\|$ are the Euclidean norms (magnitudes) of the embeddings.

The attention weight $w_{k}^{n}$ for a token $\mathbf{p}_{k}^{n}$ is computed by aggregating its cosine similarities with all tokens in other clients’ prompts. This can be expressed as:

$$ $w_{k}^{n}=\sum_{\begin{subarray}{c}k^{\prime}=1\\ k^{\prime}\neq k\end{subarray}}^{K}\sum_{n^{\prime}=1}^{N}\text{sim}(E_{k}^{n} ,E_{k^{\prime}}^{n^{\prime}})$ (6) $$

where $k^{\prime}$ iterates over all clients except the $k$-th client, $n^{\prime}$ iterates over all tokens in the prompt of client $k^{\prime}$, and $\text{sim}(E_{k}^{n},E_{k^{\prime}}^{n^{\prime}})$ is the cosine similarity between the embedding $E_{k}^{n}$ and each embedding $E_{k^{\prime}}^{n^{\prime}}$. To normalize the attention weights across all tokens in a prompt, we apply a softmax function to obtain a normalized weight $\alpha_{k}^{n}=\frac{\exp(w_{k}^{n})}{\sum_{n=1}^{N}\exp(w_{k}^{n})}$, where $\alpha_{k}^{n}$ is the normalized attention weight of the token $\mathbf{p}_{k}^{n}$. The final attention vector for all tokens in the $k$-th client’s prompt is $\bm{\alpha}_{k}=\{\alpha_{k}^{1},\alpha_{k}^{2},\ldots,\alpha_{k}^{N}\}$, where $\bm{\alpha}_{k}$ represents the normalized attention weights for all tokens in the $k$-th client’s prompt, indicating the relative importance of each token based on its semantic similarity to tokens in other prompts.

Semantic Aggregation Using High-Dimensional Clustering. After computing attention weights for all tokens, we employ high-dimensional clustering (e.g., k-means) to further filter semantically similar tokens. The clustering process proceeds as follows: The embeddings of all tokens serve as shown in Algorithm 2. To further enhance the flexibility of token selection, we employ a strategy based on embedding distance elbow detection and DBSCAN clustering. We calculate the distances between token embeddings and sort these distances, identifying significant changes as “elbow points” or inflection points. These points are used to determine the $\epsilon$ parameter for DBSCAN clustering. Subsequently, DBSCAN forms clusters based on the density and connectivity of the embeddings. This approach allows the number of clusters and the number of tokens within each cluster to be determined by the data itself, enabling adaptive and flexible grouping. Finally, the representative tokens from each cluster are reordered according to their original positions in the respective prompts, forming a consolidated global prompt. This step ensures that the global prompt remains semantically coherent and retains the most important information from each client.

Figure: Algorithm 2 Semantic Aggregation Using High-Dimensional Clustering

## 4 Evaluation

### 4.1 Evaluation Setup

Pre-trained LLMs.
In our experiments, we selected two models as backbone models: DeepSeek-V2-Lite (15B parameters) (DeepSeek-AI, ), and Llama-3.1-8B-Instruct (AI@Meta, ).

Dataset. We conducted experiments on seven datasets from the GLUE benchmark (Wang et al., ): SST-2, RTE, QNLI, MRPC, QQP, WNLI, and CoLA.
Additionally, we adopted the k-shot approach for prompt training, which will be explained in detail in the following sections. Due to the consistent number of classes across datasets, we used accuracy (ACC) instead of the Matthews Correlation Coefficient (MCC) to evaluate the prediction performance for the CoLA dataset. Similarly, for QQP and MRPC, ACC was used in place of the F1 score as the evaluation metric.

Comparison Baselines. We evaluated our pure black-box prompt-tuning federated learning method against seven state-of-the-art (SOTA) approaches. Based on the amount of information obtained about the backbone model, we categorized these methods into white-box and black-box approaches. We define white-box LLM methods as those that have access to the full parameters of the backbone model and can obtain gradient information through backpropagation.

The White-Box comparison methods include the following: FedPrompt (Zhao et al., ): A SOTA method that offers communication efficiency and privacy protection by employing a prompt exchange strategy to facilitate knowledge transfer between clients in federated learning. OpenFedLLM (Ye et al., ): An open-source research library for training large language models (LLMs) in a federated learning setting. OpenFedLLM allows for various configurations through custom FL methods and LLM replacements. In this study, we used the widely adopted FedAvg algorithm to implement federated learning for the backbone model. Manual prompt: It refers to a manually designed prompting approach based on commonly used templates for zero-shot inference.

The Black-Box LLM methods do not have access to the model’s parameters or gradients; they can only retrieve prediction outputs and the full probability distribution generated by the model during forward inference. These methods include the following: FedBiOT (Wu et al., ): This method compresses the original LLM into a lightweight model with similar performance, which is then distributed to each client. FedAvg-BBT (McMahan et al., ; Sun et al., ): A hybrid method that combines the widely used federated learning approach, FedAvg, with a black-box discrete prompt tuning method called BBT.

Implementation & Hyperparameters . The federated learning (FL) setup of our experiments follows the frameworks of FedPrompt and FedBPT. The FL environment consists of 10 clients, with a 100% client participation rate in each training round. Additionally, we adopted the few-shot learning paradigm commonly used in large-scale model research. Following the BDPL approach, for each dataset, we randomly sampled k instances from each class to form a new training set and sampled a different set of k instances to construct a new validation set. The new test set was composed of the original validation set. Detailed hyperparameter settings can be found in Appendix A.

### 4.2 Effectiveness Results

Table: Table 1: Effectiveness Results

### 4.3 Transferability Results

Table: Table 2: The Transferability Results

### 4.4 Overhead Results

Table: Table 3: Number of trainable parameters when adopting Llama 3.1 as the backbone model

### 4.5 Ablation Studies

Client-Level. To evaluate the effectiveness of the improvements made during client-level optimization, including the integration of prediction feedback loops and the use of MLM-API for prompt optimization, we conducted separate tests, as shown in Table 4. Here, Client-1 represents the approach without the feedback loop and uses random token replacement for optimization, while Client-2 only omits the feedback loop. The results in Table 4 demonstrate that the proposed client-level optimization significantly outperforms both Client-1 and Client-2 across all tasks. Specifically, our approach improves accuracy by a notable margin: for SST-2, it shows an increase of 14.6% over Client-1 and 5.6% over Client-2; for RTE, it improves by 7.1% and 4.1%, respectively. This clearly indicates the effectiveness of the feedback loop and MLM-API optimizations. Additionally, the results show that removing the feedback loop (Client-2) results in a consistent drop in performance across all tasks, confirming that integrating feedback is critical for enhancing model accuracy.

Table: Table 4: The effectiveness of our propsoed client level optimization

## 5 Conclusion

We propose FedDTPT, a FL framework that enables clients to tune discrete and transferable prompts with LLMs in black-box settings. Our approach eliminates the need for clients to access model parameters and requires only forward propagation for local training, reducing computational and storage demands for both devices and LLM service providers. Additionally, our discrete prompts are interpretable to developers and can be transferred to other LLMs without any modifications. Evaluations on several datasets using state-of-the-art PLMs show that FedDTPT outperforms existing white-box and black-box methods with significantly lower communication and memory overhead. Furthermore, FedDTPT demonstrates excellent transferability.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 3
questions: - How does the client/server handle the convergence of prompt tuning across rounds?
- Since this is a gradient-free approach, what specific criteria or metrics are used to decide whether to retain or discard a token replacement? Is there a clear definition for convergence in the feedback loop?
- Given that the clustering process groups tokens based on similarity, how does normalization of attention weights ensure that the most meaningful tokens are selected? Is there a risk that normalization skews the representation of some clusters?
- Does sharing token embeddings with the server pose any privacy risks, particularly in cases where specific tokens might reveal sensitive information? Could these embeddings be used to infer details about the client’s local data?
- One advantage of discrete prompt tuning is interpretability. Does the final global prompt maintain this interpretability after multiple aggregation rounds, or do tokens selected for optimization risk making the prompt less clear?
soundness: 3
strengths: The paper addresses a novel problem space, combining federated learning (FL), black-box prompt tuning, and discrete prompt optimization.
summary: The paper introduces FedDTPT (Federated Discrete and Transferable Prompt Tuning), a federated learning framework designed for optimizing prompts in a black-box setting with large language models (LLMs). The framework focuses on gradient-free optimization of discrete prompts, where each client independently tunes its prompt locally, and the server aggregates these prompts to create a globally optimized prompt for iterative improvement.
weaknesses: - Although the combination of black-box LLMs and FL is novel, prompt tuning as a standalone concept has been widely explored, even for discrete prompts. The contribution is incremental, focusing more on adapting existing methods (e.g., clustering and cosine similarity-based aggregation).
- For this paper to be distinctively valuable as a federated approach, it would need to clearly show how FedDTPT’s design solves  FL-specific challenges and why a centralized approach would not offer the same benefits. Thus far, it seems that the only way to solve statistical heterogeneity is to use a "small" public dataset. This may not be easily obtainable, which is why FL is needed in the first place.
- The results presented lack a clear demonstration of how FedDTPT performs under varying degrees of data heterogeneity. Centralized prompt learning could, theoretically, also use clustering or attention-based methods to aggregate prompts without needing an iterative FL framework.
- In the server optimization phase, choosing the highest-weighted token might bias the global prompt towards tokens that are slightly overrepresented, which may not always yield the best performance for all clients.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: See the weaknesses. 

Minor: The authors might want to change the color of the dark blue charts in Figure 1. It is really challenging to read the context there.
soundness: 3
strengths: 1. The authors target a more realistic setting in which the clients cannot access the embedding layer and the output distribution.

2. The authors conduct extensive experiments for evaluation.
summary: The authors introduce FedDTPT, an innovative federated learning (FL) framework designed to enable clients to tune discrete, transferable prompts with large language models (LLMs) in black-box settings. This framework offers a notable advantage by eliminating the need for client access to model parameters, relying only on forward propagation for local training, thereby significantly lowering computational and storage requirements for both devices and LLM service providers. Moreover, the discrete prompts produced by this approach are interpretable for developers and can be seamlessly transferred to other LLMs without modification. Through extensive evaluations on multiple datasets with state-of-the-art pre-trained language models (PLMs), FedDTPT demonstrates superior performance over existing white-box and black-box methods, achieving substantial reductions in communication and memory overhead. Additionally, FedDTPT exhibits impressive transferability across models.
weaknesses: 1. The presentation of local optimization is not clear. How do the clients utilize the MLM to fine-tune the local prompt?

2. In FedDTPT, the clients have access to a representative public dataset. How is this dataset constructed? Having access to this public dataset makes the comparison with the other methods unfair. The authors need to demonstrate to what extent the improvement under the non-IID setting comes from the public dataset. In addition, I do not think it will be necessary to apply FL if the public dataset is "representative."

3. FedDTPT adopts an additional language model for aggregation. It seems the aggregation will involve high computational costs.

4. It is unclear how "the representative tokens from each cluster are reordered according to their original positions in the respective
prompts" during the aggregation. It looks like the algorithm cannot guarantee the selected representative tokens correspond to different positions in the respective prompts.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: See weaknesses.
soundness: 2
strengths: 1. The proposed setting is a novel problem which has not been studied before.

2. Experimental baselines include many of the recently FL-LLM frameworks, such as OpenFedLLM,FedPrompt.
summary: This paper proposes a discrete and transferable prompt tuning for black-box large language models. It adopts a token-level discrete prompt tuning strategy on the client side and an embedding-based clustering techniques on the server. Experimental results demonstrate the effectiveness of the proposed method.
weaknesses: 1. The writing and presentation of the paper is often unclear and convoluted, making it difficult to follow the author's argument. 
a) e.g. Eq 1's notations are inconsistent with those explained in the text, e.g. \mathcal(P) vs P, and some are left undefined, e.g. L, w. In Algo 1, it is unclear what "modifications" or "feedback_info" represents as they are not properly introduced.  It is unclear what "MLM API" represents in this paper. at one place, it refers to the "pre-trained language models like BERT", at other places such as in the figure, it appears to use "LLM API"  as MLM. Due to these representation issues, it is hard to understand from the context or figure how exactly the local optimization performs. In the experiment section, it is unclear how is "transferability" defined and how are the experiments on that performed.  In Table 2, it is unclear what is "D to L" or "L to D".

2. The technical novelty and contribution is a bit limited. Both the local optimization part and the central clustering part appear to be straightforward. If I understand correctly,  the search space for the optimized prompts are only 3 to 4 prompts (according to appendix), and it is quite obvious from the human experience to identify the best from these prompts. Therefore I am not convinced that the contribution of the proposed method is significant. It is advised that the authors provide more challenging examples on the local prompts and the resulted aggregated ones to illustrate the importance of the prompt optimization and clustering steps. Also, it is suggested that the authors discuss on the technical challenges, particularly in the FL setting with non-iid data to demonstrate the technical contribution.

3. Experimental results are missing important details, making them unconvincing. First of all, it is surprising to see that Black-boxed LLM with discrete prompts outperforms parameter fine-tuning approaches (white-boxed) across specific tasks, and by a large margin. It is unclear how are the white-boxed baselines are performed or sufficiently trained, and there are no sufficient details for the baselines. For the manual approach, it is also unclear exactly how the prompts are selected and distributed across clients. FedBiOT is listed as a baseline but no results are found.

---

confidence: 3
contribution: 1
initial_rating: 1
presentation: 1
questions: NA
soundness: 1
strengths: 1) This paper seems to provide an algorithm for federated prompt tuning.
2) Some preliminary experiment results are provided to demonstrate the effectiveness of the proposed algorithm.
summary: This paper proposes a federated learning method for tuning the prompts for LLM. The key idea seems to be: 1) the local clients optimize prompts by updating local prompts with local accuracy results; 2) the central server aggregates the prompts and computes the embedding of the tokens in the prompt, and uses some aggregation results to provide some updated global prompt for the local client. Some experimental results are provided to show the effectiveness of the algorithm.
weaknesses: This paper is a draft far from even being reviewed in the conference because of its writing.
1. It is questionable whether Section 3.4 is finished. 1) In the last paragraph of Page 6, it mentions some steps requiring the DBSCAN algorithm, but DBSCAN never appears in Algorithm 2. Besides, it is unclear how k-means and DBSCAN work sequentially with the text description. 2) Section 3.4 stops after introducing their algorithm, but it is unclear how the server generates the global prompt or shares global updated information with the clients in FL. 
2. It is super confusing what the "MLM" is in the paper. It seems MLM is an important method/API used in the algorithm, but it is never spelled out until Page 5 despite being mentioned multiple times before that. Even within Page 5, MLM is used to refer to two different things: masked language models and (embedding layers of?) pre-trained language models. It is also mentioned that the client can use MLM API, but the input and output of such API are unclear in the paper.
3. The experiment part is also unclear and probably unfinished. 1) For example, the authors mention FedBiOT as a black-box method when introducing their baseline, but no results of this method are shown in the table. 2) The experimental results of the baseline methods seem significantly lower than the reports in other papers (e.g., results of the baseline on SST-2), which may bring doubts about whether those algorithms are implemented faithfully.