When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario
Year: 2023

Abstract:
Large pre-trained language models (PLMs) have garnered significant attention for their versatility and potential for solving a wide spectrum of natural language processing (NLP) tasks. However, the cost of running these PLMs may be prohibitive. Furthermore, PLMs may not be open-sourced due to commercial considerations and potential risks of misuse, such as GPT-3. The parameters and gradients of PLMs are unavailable in this scenario. To solve the issue, black-box tuning has been proposed, which utilizes derivative-free optimization (DFO), instead of gradient descent, for training task-specific continuous prompts. However, these gradient-free methods still exhibit a significant gap compared to gradientbased methods. In this paper, we introduce gradient descent into black-box tuning scenario through knowledge distillation. Furthermore, we propose a novel method GDFO, which integrates gradient descent and derivative-free optimization to optimize task-specific continuous prompts in a harmonized manner. Experimental results show that GDFO can achieve significant performance gains over previous state-ofthe-art methods.

heading: 1 Introduction
text: Large pre-trained language models (PLMs) (Devlin et al., 2019; Liu et al., 2019; Yang et al., 2019; Raffel et al., 2020) have attracted considerable attention for their versatility and potential for solving a wide spectrum of Natural Language Processing (NLP) tasks. Especially, through prompt-based learning (PL) (Liu et al., 2021a; Gu et al., 2022), PLMs have consistently demonstrated impressive performance on various downstream tasks with a
∗Equal contribution. †Corresponding author. ‡ Shanghai Engineering Research Center of Big Data
Management
few labeled samples. However, it is a challenge to extend the benefits of these large PLMs to a broader audience. For users, the cost of running these models may be prohibitive; for service providers, they may not open source the model parameters due to commercial considerations and potential risks of misuse1. One possible solution is to deploy PLMs as a service, enabling users to access the advanced capabilities of PLMs through their inference APIs, such as GPT-3 (Brown et al., 2020), ERNIE (Sun et al., 2021) and Yuan (Wu et al., 2021b).
In this scenario, the large pre-trained language model provided by the server is considered as a black box. In order to perform various downstream tasks, users are required to construct task-specific prompts or select training samples (Brown et al., 2020) to input into the black box. We can manually construct discrete prompts, which are simple and effective but may not fully utilize training data,
1https://openai.com/blog/openai-api/
868
potentially resulting in suboptimal performance on some tasks. Instead of designing hand-crafted discrete prompts, there have been an increasing number of studies on continuous prompt tuning (Lester et al., 2021; Liu et al., 2021a; Ding et al., 2022), which aim to train continuous prompts and add them to the original samples. Trainable continuous prompts have also shown remarkable success on various tasks, but most existing methods optimize the continuous prompts through back-propagation, which is unavailable in the black-box scenario. To solve the issue, Sun et al. (2022b) have recently proposed Black Box Tuning (BBT), which utilizes random projection matrices and derivative-free optimization (DFO) (Kolda et al., 2003; Conn et al., 2009; Rios and Sahinidis, 2013), instead of gradient descent, for training continuous prompts in the black-box scenario. Built upon BBT, BBTv2 (Sun et al., 2022a) prepends continuous prompts to each layer of the PLM and further presents a divide-andconquer gradient-free algorithm to alternately optimize the prompts at different layers. Both BBT and BBTv2 have shown their superiority against other gradient-free methods. Despite the success, there remains a significant gap compared to gradientbased methods on certain tasks. For example, compared against BBTv2, Adapter (Houlsby et al., 2019), a gradient-based method, leads by 4.35% on the DBPedia dataset (as shown in Figure 1). Therefore, we consider that the incorporation of gradient descent into the black-box scenario may potentially enhance the performance of the model.
Based on the insights discussed above, in this paper, we introduce gradient descent into the blackbox scenario through knowledge distillation techniques. In particular, we propose a novel approach named GDFO to combine Gradient descent with Derivative-Free Optimization, allowing them to jointly optimize task-specific continuous prompts. First, we adopt the technique of knowledge distillation, where a student model is trained to emulate the knowledge of the black-box model, referred to as the teacher model. Then, a prompt generator is trained by gradient descent through the student model, while utilizing derivative-free optimization algorithms to optimize continuous task-specific prompts. The continuous prompts generated by the prompt generator and the prompts optimized by the derivative-free algorithm are further integrated to serve as the final prompts. Finally, we perform extensive experiments on seven benchmark datasets
to show that GDFO can achieve significant performance gains over other state-of-the-art methods. The main contributions of the paper are summarized as follows:
• To the best of our knowledge, we are the first to utilize gradient descent to optimize taskspecific continuous prompts in the black-box scenario through knowledge distillation.
• We propose a novel method GDFO, which integrates gradient descent and derivative-free optimization to optimize task-specific continuous prompts in a harmonized manner.
• We conduct comprehensive experiments on seven benchmark datasets under the black-box scenario. Empirical results demonstrate the superiority of GDFO over other competitors.

heading: 2 Related Work

heading: 2.1 Prompt-based Learning
text: Prompt-based learning, in which the PLM is adapted to various tasks by task-specific prompts, has emerged as a promising framework. Brown et al. (2020) shows that PLM can perform excellently in few-shot learning by using manual prompts concatenated with samples. However, designing prompts in a hand-crafted fashion requires substantial time and experience and may not find the optimal ones (Jiang et al., 2020; Shin et al., 2021). To solve the problem, researchers attempt to use automated prompts. LM-BFF (Gao et al., 2021) uses prompt-based fine-tuning with automatically searched prompts and generates task demonstrations to be a part of the input context. P-tuning (Liu et al., 2021b) optimizes the continuous prompts using gradient descent as an alternative to discrete prompt searching. P-tuning v2 (Liu et al., 2021a) adopts continuous prompts for each layer of the PLMs to improve the model performance. Prefixtuning (Li and Liang, 2021) optimizes continuous task-specific vectors and prepends them to the input texts. Input-tuning (An et al., 2022) fine-tunes both the continuous prompts and the input representations, leading to a more effective way to adapt unfamiliar inputs to frozen PLMs.

heading: 2.2 Black-box Tuning
text: Due to commercial considerations, large PLMs such as GPT-3 (Brown et al., 2020) are only pro-
vided as a service in the cloud, resulting inaccessible parameters and gradients of PLMs. To tackle this issue, BBT (Sun et al., 2022b; Diao et al., 2022) has been proposed to optimize the continuous prompts via derivative-free optimization (DFO). As an improved version of BBT, BBTv2 (Sun et al., 2022a) inserts prompts to each layer of the PLMs instead of optimizing the prompt merely in the input layer. Furthermore, GrIPS (Prasad et al., 2022) proposes a gradient-free search approach to generate discrete prompts. Besides, RLPrompt (Deng et al., 2022) optimizes discrete prompts through reinforcement learning and utilizes a continuous policy network which is highly parameter-efficient to generate prompts. PALP (Cho et al., 2022) combines linear models and in-context learning (Brown et al., 2020) to augment training samples with the templates for better contextualization. To improve the computational efficiency, PromptBoosting (Hou et al., 2022) constructs a pool of prompts via a gradient-free approach and ensembles many weak learners using the ADABOOST algorithm to enhance the model performance. Despite the success of the above approaches, all of them do not optimize continuous prompts through gradient descent (GD) in the black-box scenario, our method introduces GD to the scenario through knowledge distillation and combines GD and DFO to jointly optimize continuous prompts, which provides a novel insight for future black-box tuning approaches.

heading: 2.3 Knowledge Distillation
text: As a representative method of model compression, knowledge distillation transfers the knowledge from a larger deep neural network (teacher) to a smaller network (student) (Hinton et al., 2015; Kim and Rush, 2016). There have been different distillation algorithms being proposed to face more complex settings of transferring knowledge, including adversarial distillation (Ma et al., 2020; Wang et al., 2022), multi-teacher distillation (Guan et al., 2020; Yuan et al., 2021) and data-free distillation (Fang et al., 2022; Binici et al., 2022). Furthermore, the superior success of PLMs has also spurred researchers to distill PLMs into smaller models while retaining performance. DistilBERT (Sanh et al., 2019) introduces a triple loss combining language modeling and cosine-distance losses to leverage the inductive biases learned by large models during pre-training. TinyBERT (Jiao et al., 2020) performs a Transformer distillation method at both the pre-training and task-specific learning stages. NewsBERT (Wu et al., 2021a) designs a collaborative learning framework where the student model can learn from the experience of the teacher model. In this paper, we consider knowledge distillation to transfer knowledge from a black-box teacher
to a student, which is used for training a prompt generator by gradient descent.

heading: 3 Method
text: In this section, we describe our approach GDFO. We first give an overview of GDFO, which is illustrated in Figure 2. GDFO first trains a student model by aligning its outputs to that of the teacher model (i.e., the black-box model). Then, GDFO trains the prompt generator by gradient descent while simultaneously optimizing the continuous prompts via DFO. Finally, the final prompts are obtained by integrating the prompts generated by the prompt generator with those optimized by DFO, which are then fed into the black-box model together with query instances to obtain predictions. Next, we describe each component of GDFO in detail.

heading: 3.1 Knowledge Distillation
text: Given a student model S and a teacher model T , the objective of knowledge distillation (KD) is to enhance the performance of S by aligning its outputs with those of T , which is accomplished by reducing the divergence between the probability distributions generated by S and T . In the blackbox scenario, the black-box model is considered as T . We utilize T ’s outputs as soft targets for S to learn. Given a training instance, we randomly select n tokens from the PLM vocabulary to construct a random prompt pr, which is concatenated to the beginning of the instance. Additionally, a hand-crafted template2 is appended to the end of the instance. We use the concatenated sentence as the input x. We denote S(x) and T (x) as the output logits of S and T for input x, respectively. The KD can be conducted by minimizing the KullbackLeibler (KL) divergence distance between the student and teacher predictions:
LKL = KL(σ(S(x)/τ)∥σ(T (x)/τ)), (1)
where σ(·) denotes the softmax function and τ is a temperature hyper-parameter. The student parameters are updated according to LKL and the cross-entropy loss LCE over the ground-truth y:
L = (1− λ)LCE + λLKL, (2)
where λ is a weight and LCE is defined as:
LCE = −y log σ(S(x)). (3) 2The details of templates are shown in Table 1.
Algorithm 1 Training Procedure Input: Training data {Xtrain,Ytrain}; Black-box
model T ; Student model Sθ; Prompt generator Gµ; Epochs for knowledge distillation Ekd; The number of API calls N ; The PLM vocabulary V; Hand-crafted template t; # Knowledge Distillation
1: for each i ∈ Ekd do 2: for each x ∈ Xtrain do 3: pr ← Random_Sample(V, n); 4: ŷT , ŷS ← T ([pr;x; t]), Sθ([pr;x; t]); 5: Calculate L by Equation 2; 6: Update θ by L; 7: end for 8: end for
# Prompt Generator 9: for each i ∈ N do
10: for each x ∈ Xtrain do 11: pGD ← Gµ(x); 12: Get z by CMA-ES; 13: Get p by Equation 4; 14: ŷT , ŷS ← T ([p;x; t]), S([p;x; t]); 15: Calculate LTCE ,LSCE by Equation 3; 16: Update µ by LSCE ; 17: Optimize CMA-ES by LTCE ; 18: end for 19: end for

heading: 3.2 Prompt Generator
text: Upon the completion of training S via knowledge distillation, the student parameters are frozen and a prompt generator is optimized by gradient descent with the purpose of generating continuous prompts pGD ∈ RD for given samples. Meanwhile, following BBT (Sun et al., 2022b), we optimize intermediate vector z ∈ Rd through CMAES (Covariance Matrix Adaptation Evolution Strategy) (Hansen and Ostermeier, 2001; Hansen et al., 2003), which is a widely used evolutionary algorithm for non-convex black-box optimization in continuous domain. Then a random projection matrix A ∈ RD×d is utilized to project z into the highdimensional space. Finally, we randomly sample n tokens from the PLM vocabulary as initial prompt p0 and get final continuous prompt p ∈ RD:
p = αpGD + (1− α)(p0 +Az), (4)
where α is a balancing weight. Further information regarding the initialization of A and the specifics
of the optimization procedure of CMA-ES can be found in Sun et al. (2022b). Given a training instance, p is concatenated to the beginning of it and a hand-crafted template2 is appended to the end of it. The concatenated sample is fed into S and T . Then the output logits are obtained and used to compute LCE , which is utilized to update the parameters of the prompt generator and optimize z through CMA-ES. The overall training procedure of GDFO is summarized in Algorithm 1.

heading: 3.3 Inference
text: During the inference stage, given a query instance, we first input it into the prompt generator to generate pGD. Subsequently, we combine pGD, p0, and Az that have been optimized through CMAES to obtain the final continuous prompt p through the Equation 4. Next, similar to the training stage, we concatenate p to the front of the query instance and append the hand-crafted template2 to the end of it. Finally, we input the concatenated sample to the black-box model to obtain the prediction. The overall inference procedure is shown in Figure 3.

heading: 4 Experiments
text: In this section, we perform comprehensive experiments to compare our proposed model with twelve competitive baselines on seven downstream tasks.

heading: 4.1 Datasets
text: We perform experiments on a variety of language understanding tasks, including sentiment analysis, topic classification, natural language inference (NLI), and paraphrasing. Statistics of these datasets are given in Table 1. Specifically, we utilize the following datasets:
Sentiment analysis: SST-2 (Socher et al., 2013) and Yelp polarity (Zhang et al., 2015) consist of
text samples with assigned sentiment labels (e.g. positive or negative).
Topic classification: AG’s News (Zhang et al., 2015) and DBPedia (Zhang et al., 2015) contain text samples with pre-defined topics.
NLI: SNLI (Bowman et al., 2015) and RTE (Wang et al., 2018) are composed of sentence pairs and the objective is to determine the relationship between the two sentences, such as entailment, contradiction and neutral.
Paraphrase: MRPC (Dolan and Brockett, 2005) contains sentence pairs and the goal is to recognize semantic equivalence between the two sentences.

heading: 4.2 Baselines
text: We compare GDFO with twelve competitive methods, which can be grouped into two categories: gradient-based methods and gradient-free methods.
For gradient-based methods, we consider six baselines: (1) Model Tuning fine-tunes the entire PLM through training data. (2) Adapter (Houlsby et al., 2019) is a new module added between layers of a PLM. The parameters of the original network remain fixed, yielding a high degree of parameter sharing. (3) BitFit (Zaken et al., 2022) is a sparsefinetuning method where most of the network parameters are frozen and only the bias-terms of the model (or a subset of them) are being modified. (4) LoRA (Hu et al., 2021), an efficient adaptation strategy, allows us to train some dense layers in a neural network indirectly by optimizing rank decomposition matrices of the dense layers’ change, while keeping the pre-trained weights frozen. (5) Prompt Tuning (Lester et al., 2021) freezes the entire PLM and only allows additional tunable tokens to be prepended to the input text. (6) P-Tuning v2 (Liu et al., 2021a) applys continuous prompts for every layer of the PLM instead of the mere input layer.
For gradient-free methods, we also consider six baselines: (1) Manual Prompt conducts subsequent experiments using hand-crafted prompts following the pre-defined templates in Table 1. (2) In-Context Learning (Brown et al., 2020) provides a few training examples for the model to improve its capability of few-shot learning. (3) Feature-MLP trains a two-layered MLP classifier provided with embeddings encoded by the PLM. (4) Feature-BiLSTM trains a bidirectional LSTM on the word representations and connects it to a
linear classifier. (5) BBT (Sun et al., 2022b) optimizes the continuous prompt prepended to the input text via derivative-free optimization (DFO). (6) BBTv2 (Sun et al., 2022a) proposes a divideand-conquer algorithm to alternately optimize the prompt at each layer of the PLM. Compared with BBT, BBTv2 inserts prompts to each layer of the PLM instead of optimizing the prompt merely in the input layer.

heading: 4.3 Implementation
text: Few-shot setting We adopt the same procedure as described in previous studies (Zhang et al., 2020; Sun et al., 2022a) to establish a true few-shot learning environment. Specifically, we randomly select k samples per class to create a k-shot training set Dtrain, and form a development set Ddev by randomly selecting another k samples from the original training set, resulting in |Dtrain| = |Ddev|. We use the original development sets as our test sets Dtest. For datasets that do not have development sets, we use the original test sets. It is noted that |Dtest| ≫ |Dtrain| = |Ddev|.
Experimental settings To compare with BBTv2 (Sun et al., 2022a), we mainly use RoBERTaLARGE (Liu et al., 2019) as the blackbox model. For hyper-parameters, we use the grid search to find the best for our model. For knowledge distillation, we use BERTLARGE (Devlin et al., 2019) as our student model. We set the temperature τ to 1 and the balancing weight λ to 0.5. We fine-tune the student model for 2,000 epochs with the learning rate 1e− 4. For prompt generator, we use a fully connected layer and set the dimensionality of the fully connected layer to 1024. The learning rate of the prompt generator is 1e − 5. For CMA-ES, following Sun et al. (2022b), we set the prompt length n to 50. The dimensionality of z is set to 500 and
the population size of CMA-ES is set to 20. The balancing weight α is set to 0.5. We train our prompt generator and run CMA-ES for 8,000 API calls. All baseline results are recorded in Sun et al. (2022a). We run all the experiments on a single NVIDIA v100 GPU.

heading: 4.4 Main Results
text: The results of 16-shot setting on various downstream tasks are shown in Table 2. From the table, GDFO consistently outperforms all the baselines on the average performance. Specifically, in the gradient-based comparison, GDFO achieves an average accuracy of 81.85%, which outperforms the runner-up gradient-based model, LoRA, by a notable 3.84% improvement. When compared against the gradient-free methods, GDFO leads BBTv2 by 5.26% and 3.89% on the SNLI and RTE datasets, respectively. Our model generates a continuous prompt for each sample, rather than using an optimized continuous prompt for all samples, such as BBT and BBTv2. Furthermore, the incorporation of both DFO and gradient descent during the training stage allows GDFO for more comprehensive and efficient training of continuous prompts, resulting in a notable improvement in the model performance.

heading: 4.5 Ablation Study
text: We conduct an ablation study to investigate the characteristics of the main components of GDFO. As illustrated in Figure 4, the results3 demonstrate that GDFO outperforms GDFO-w/o-KD. For instance, on the SNLI dataset, the accuracy of GDFO is 62.53%, whereas that of GDFO-w/o-KD is only 58.51%. This indicates that the knowledge distillation module, which transfers the knowledge of the teacher model to the student model by aligning
3The evaluation metric used in the ablation study is F1 score for MRPC and accuracy for other datasets.
the outputs of the student model with those of the teacher model, effectively improves the model performance. Additionally, when removing derivativefree optimization, a significant decline is observed across all datasets, with an average decrease of 6.5%. This demonstrates the effectiveness of incorporating derivative-free optimization in the blackbox scenario. It is worth noting that when removing the prompt generator, the student model will not function, which means that gradient descent is eliminated. In this case, our method degrades to a gradient-free method BBT. The results, as shown in Table 2, reveal that GDFO achieves significant performance gains over BBT across all datasets, which demonstrates the effectiveness of training the prompt generator through gradient descent in the black-box scenarios.

heading: 4.6 Analysis
text: Different Black-Box Models To evaluate the universality of GDFO across PLMs with varying architectures, in addition to encoder-only PLMs (e.g.,
RoBERTaLARGE), we conduct experiments using decoder-only (e.g., GPT-2LARGE) and encoderdecoder PLMs (e.g., BARTLARGE and T5LARGE) as black-box models. As shown in Figure 5, GDFO achieves superior performance over other competitors across all the settings. For example, When using GPT-2 as the black-box model, GDFO achieves 87.5% and 85.2% on the SST-2 and DBPedia datasets, respectively. In particular, it outperforms BBT by a notable 11.9% and 15.5% improvements in both cases. When considering BART as the black-box model, GDFO leads BBTv2 by 8.12% on the DBPedia dataset. All the results clearly show the generalizability of our model across various PLMs.
Different Student Models We next conduct an in-depth experiment for student models on three datasets. The results are shown in Table 3. From the results, different student models have a impact on the performance of GDFO (approximately 2% on average). Furthermore, we observe that student
models whose architectures are similar to the blackbox model tend to exhibit superior performance. For instance, when both the black-box model and the student model are RoBERTaLARGE (Liu et al., 2019), GDFO achieves the best performance. When comparing models with identical architectures, such as BARTLARGE (Lewis et al., 2020) and T5LARGE (Raffel et al., 2020), T5 exhibits superior performance, which may be due to the fact that the T5 model has twice the number of parameters as the BART model. The increased capacity allows the T5 model to better capture and represent the relationships within the input data, resulting in improved performance.
Effect of Balancing Weight The balancing weight α plays a crucial role in determining the performance of the model by controlling the influence of pGD and Az. As the value of α increases, the influence of pGD becomes more prominent, while conversely, as the value of α decreases, the influence of Az becomes more pronounced4. As illustrated in the Figure 6, when α is set to an extreme value, either too large or too small, it tends to have a negative impact on the model performance. We observe that the average performance of the model across three datasets is optimal when α is set to 0.5, further emphasizing the importance of the combination of derivative-free optimization and gradient descent in improving the performance of the model.

heading: 5 Conclusion
text: In this paper, we introduced gradient descent into the black-box tuning scenario through knowledge distillation for the first time, which provided a novel insight for future black-box tuning approaches. Furthermore, we proposed a novel method, GDFO, which integrates gradient descent and derivative-free optimization for jointly training continuous prompts. GDFO first trains a student model to enhance the performance by aligning its outputs with those of the teacher model (i.e., the black-box model). After that, GDFO trains a prompt generator using gradient descent while
4p0 is fixed, thus its effect on the model performance is disregarded in the analysis.
simultaneously optimizing a continuous prompt using DFO algorithm. Experimental results on various datasets show that GDFO can achieve significant performance gains over other gradient-free and gradient-based methods.
Limitations
We summarize the limitations of this work as follows: (1) We conduct experiments on 7 language understanding tasks across 4 types (i.e., sentiment analysis, topic classification, natural language inference and paraphrasing). However, the effectiveness of GDFO on tasks such as sequence labeling and generation tasks has yet to be fully examined. (2) Our proposed method uses a student model and a prompt generator, thereby resulting in a higher computational resource requirement in comparison to gradient-free methods. Therefore, it may not be suitable for implementation on certain edge devices, but it is more appropriate for personal or enterprise users who have access to a certain degree of computational resources and have stringent requirements for the model performance. (3) We only focus on the few-shot setting in this paper. It is possible to extend our work to other scenarios such as semi-supervised learning and we will further explore it in the future research.
Ethics Statement
The proposed method has no obvious potential risks. All the scientific artifacts used/created are properly cited/licensed, and the usage is consistent with their intended use.

heading: Acknowledgements
text: This work has been supported by the National Natural Science Foundation of China under Grant No. U1911203, and the National Natural Science Foundation of China under Grant No.61977025.

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: The Section of Limitation

heading: 3 A2. Did you discuss any potential risks of your work?
text: The Section of Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: The Section of Abstract and Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 4

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Section 4

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Section 4 and The Section of Ethics Statement

heading: 3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any
text: information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The Section of Ethics Statement

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4
C 3 Did you run computational experiments? Section 4

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Section 4

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Not applicable. Left blank.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Not applicable. Left blank.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not applicable. Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: In this paper, we introduced gradient descent into the black-box tuning scenario through knowledge distillation for the first time, which provided a novel insight for future black-box tuning approaches. Furthermore, we proposed a novel method, GDFO, which integrates gradient descent and derivative-free optimization for jointly training continuous prompts. GDFO first trains a student model to enhance the performance by aligning its outputs with those of the teacher model (i. e. , the black-box model). After that, GDFO trains a prompt generator using gradient descent while
4p0 is fixed, thus its effect on the model performance is disregarded in the analysis. simultaneously optimizing a continuous prompt using DFO algorithm. Experimental results on various datasets show that GDFO can achieve significant performance gains over other gradient-free and gradient-based methods. Limitations
We summarize the limitations of this work as follows: (1) We conduct experiments on 7 language understanding tasks across 4 types (i. e. , sentiment analysis, topic classification, natural language inference and paraphrasing). However, the effectiveness of GDFO on tasks such as sequence labeling and generation tasks has yet to be fully examined. (2) Our proposed method uses a student model and a prompt generator, thereby resulting in a higher computational resource requirement in comparison to gradient-free methods. Therefore, it may not be suitable for implementation on certain edge devices, but it is more appropriate for personal or enterprise users who have access to a certain degree of computational resources and have stringent requirements for the model performance. (3) We only focus on the few-shot setting in this paper. It is possible to extend our work to other scenarios such as semi-supervised learning and we will further explore it in the future research