SPC: Soft Prompt Construction for Cross Domain Generalization
Year: 2023

Abstract:
Recent advances in prompt tuning have proven effective as a new language modeling paradigm for various natural language understanding tasks. However, it is challenging to adapt the soft prompt embeddings to different domains or generalize to low-data settings when learning soft prompts itself is unstable, task-specific, and bias-prone. This paper proposes a principled learning framework—soft prompt construction (SPC)—to facilitate learning domainadaptable soft prompts. Derived from the SPC framework is a simple loss that can plug into various models and tuning approaches to improve their cross-domain performance. We show SPC can improve upon SOTA for contextual query rewriting, summarization, and paraphrase detection by up to 5%, 19%, and 16%, respectively.

heading: 1 Introduction
text: Prompting has emerged as a new paradigm for tuning language models (LMs), drawing increasing research attention. It shows impressive utility in stimulating an LM’ hidden knowledge without finetuning it (Petroni et al., 2019). Simply adding some descriptive words to the input can steer a generalpurpose LM to a specific task in a zero-shot fashion. Table 1 shows some examples of prompting. The prompt “in other words” is added after the conversation as additional input, turning the LM into a contextual query rewriter. Any natural language understanding task can be reformulated into a generation task with proper prompting (Liu et al., 2021b; Raffel et al., 2020). As shown in Table 1, the dialogue state tracking task is converted to generating the slot value given the prompted input. As a result, a natural language understanding task turns into a prompt design task. More sophisticated prompting techniques are derived to better adapt the LM, such as in-context learning (Xie et al., 2021), chain-ofthought (Wei et al., 2022). Researchers hypothesize
that these prompting techniques make the LM understand the task instructions and do implicit task adaptation (Xie et al., 2021; Wang et al., 2022a).
Albeit the impressive utility of these prompting techniques, they have some limitations that encourage further research. One major drawback is that the prompt templates significantly impact model performance but require handcrafting, which is why there is increasing prompt engineering and sharing of “good” templates (Zhou et al., 2022). Chosen unwisely, a prompt template would not work or even hurt the model inference (Liu and Chilton, 2022; Wang et al., 2022b). Further, one has to design prompts for each task, making it inefficient and less robust to transfer across models or tasks (Sanh et al., 2022). The other drawback of prompting is that it works best for large models (Reynolds and McDonell, 2021). However, training and serving large LMs are known to be costly.
These limitations drive us to study new prompting methods that require little manual effort and work for large and small models. Soft prompting is one of these approaches. Unlike prompting that adds words to the input, soft prompting adds embeddings to the model, such as injecting embeddings to the token embedding layer or other model layers (Asai et al., 2022). Such embeddings, or soft prompts, can be automatically learned. Many studies have shown that soft prompting works well for smaller-sized models and can be as effective as fine-tuning large LMs with orders of magnitude fewer trainable parameters (Lester et al., 2021; Li and Liang, 2021; Liu et al., 2021c; Su et al., 2022).
Soft prompting is a promising approach to further improve cross-domain generalization and task adaptation for all-sized models. However, some barriers need to be removed. First, learning soft prompts lacks guidance, and traditional learning objectives (such as cross-entropy) often lead to unstable training, sub-optimality, and unstable performance. Further, the unstable performance is am-
118
plified in cross-domain settings. Soft prompts are often biased by a specific domain and hence less effective for other domains. In low-data settings, it is particularly difficult to learn soft prompts with consistent cross-domain performance (Su et al., 2022; Lester et al., 2021; Liu et al., 2021b; Chen et al., 2022a).
Our objective is to derive a principled paradigm for learning domain-adaptable soft prompts, for which we propose the soft prompt construction (SPC) framework. Using this SPC framework, we further derive a simple loss: the SPC loss. The loss supervises the optimization of soft prompts without re-tuning the entire model. The learned soft prompts demonstrate superior cross-domain generalization than other fine-tuning and prompt tuning approaches. Our contributions are the following:
(1) We propose an SPC framework that learns soft prompts in a zero/few-shot manner and demonstrates superior cross-domain generalization in extensive experiments on diverse tasks.
(2) We propose an SPC loss that can simply plug into various language models and boost their in/cross-domain performance.
(3) Our SPC framework builds upon the simplest soft prompt tuning approach, requiring orders of magnitude fewer trainable parameters than finetuning or other soft prompt tuning approaches.

heading: 2 Related Work
text: Prompting adds token-level templates to the model input, often designed manually (Petroni et al., 2019; Liu et al., 2021b). AutoPrompt automates prompt design via gradient-based search (Shin et al., 2020). Other approaches like (Gao et al., 2021; Sun et al., 2021) may use LMs to generate the prompts. Two prompting techniques are particularly useful for large LMs (LLMs): in-context learning (ICL) and chain-of-thought (CoT). ICL uses a few examples
to demonstrate the task, and the LLM can perform the task in a few-shot manner (Xie et al., 2021). CoT instructs an LLM to reason step-by-step to perform a task (Wei et al., 2022). CoT can be seen as a special case of ICL as it often provides examples to demonstrate step-by-step reasoning, such as ReAct (Yao et al., 2022), self-consistency (Wang et al., 2022a). Token-based prompts’ performance is highly dependent on prompt selection and constraint on natural language words.
On the other hand, soft prompt tuning uses embedding-level prompts and is optimized by gradients. The prompt embeddings do not necessarily represent natural language words and can generalize to different domains. DART (Zhang et al., 2021) treats input token embeddings as prompt templates. Qin and Eisner (2021) optimize soft prompts via perturbing a prompt mixture. PrefixTuning (Li and Liang, 2021) prepends prompt embeddings to the model layers and uses additional networks to re-parameterize them. P-Tuning v2 (Liu et al., 2021c) extends PrefixTuning and achieves performance comparable to fine-tuning. Such works often use deep prompts (adding prompt embeddings to all model layers) or reparameterization, introducing a large number of new parameters (Li and Liang, 2021; Jiang and Wang, 2022). Lester et al. (2021) add task-specific soft prompts to the model input and obtain performance comparable to finetuning large LMs, especially on low-resource domain adaptation tasks.
However, soft prompts’ performance is unstable and sensitive to initialization (Qin and Eisner, 2021). Learning soft prompts across tasks is challenging, especially in low-data scenarios (Gu et al., 2022). To this end, Chen et al. (2021) construct soft prompts via knowledge injection and implicit relation constraint. Gu et al. (2022) pretrain soft prompts on varied tasks to improve generalization but require prompts to be added during pre-training. Qin et al. (2021) study the generalization of soft prompts via (nonlinear) subspace decomposition. However, Su et al. (2022) find that the prompt dis-
tance cannot be appropriately quantified by Euclidean or cosine distances, implying a nonlinear and (potentially) non-metrizable prompt embedding space. Hence, constructing the prompt embedding space via decomposition is challenging.

heading: 3 Soft Prompt Construction
text: Using the examples in Table 1, prompting requires us to design different templates for different tasks, such as “in other words” for contextual query rewriting and “TL;DR” for summarization. Such handcrafting process is inefficient, does not generalize, and the performance is unstable. We aim to replace these prompt tokens with soft prompt embeddings and automatically learn domain-adaptable soft prompts. Instead of designing prompt templates, we devise a method to learn prompt embeddings that are universal to all tasks. Such prompt embeddings aim to align the model s.t. [Input] + [Prompt] yields [Output]; e.g., [Input] In other words –> [Output] ⇒ [input token embeddings][prompt embeddings] –> [output embeddings]. We present a soft prompt construction (SPC) framework for learning cross-domain generalizable soft prompts.
Consider the data on cross-domain tasks D = X× Y consisting of the input set X and output set Y. For text generation tasks, for example, each element in X or Y represents a sequence of tokens. Given the generality of the tasks and data structures, we omit the dimensionality in the notations when there is no confusion. The objective of a language model is to maximize the expected log-likelihood
maxEFy [log fθ(y | x)] (1)
where x ∈ X, Fy is the probability measure w.r.t. random variable y (note we use different fonts for a random variable y and a sample y), and fθ is the likelihood function with model parameter θ and the conditional density f(y | x). To help this objective,
prompting is introduced and prompts p are added to the input [p, x]
maxEFy [ log ∫
p fθ(y | p, x)dνp
] (2)
where νp is the probability measure of the prompt, and we will omit this for notional simplicity when there is no confusion. Let L = log ∫ p fθ(y | p, x). We have
L = log
∫
p fθ(y | p)fθ(p | x) (3)
≥ ∫
p [log fθ(y | p) + log fθ(p | x)] (4)
where from (3) to (4) we apply Jensen’s inequality. To maximize (2), we need to find a maximizer of the lower-bound (4). Taking the partial derivative of (4) w.r.t. p and setting it to zero, one of the feasible maximizers p∗ is obtained by (5)
p∗ = x⊕ y (5) where ⊕ means direct sum. Indeed, if we plug (5) into (4), it achieves the maximal (3). When using soft prompts [p, x] fθ7→ y ⇒ [ep, ex] fθ7→ ey where ex, ey, ep are embeddings of x, y, and p, relation (5) naturally extends to the embedding representations
e∗p = ex ⊕ ey (6) We propose to enforce such a relation using an SPC loss
lspc = ∥W [Wxex;Wyey]−Wpep∥2 (7) where [·; ·] is embedding concatenation, W s are linear transformations that will be learned during model training. The SPC loss is directly added to the original loss (e.g., cross-entropy) to supervise the training.
For those interested, we also provide an algebraic view for our methodology in Appendix B.
Implementation details A backbone language model is used and its parameters are frozen. The prompt embeddings ep have fixed length and are concatenated to the input token embeddings ex, effectively increasing the input length. Prompt length and concatenation are explained in Section 4. Output embeddings ey are taken from the last output layer. During training, the SPC loss (7) is directly added to the original loss (e.g., cross-entropy loss). The prompt embeddings and the linear weights of the SPC loss are learned. At inference time, the linear weights are discarded, and only the learned prompt embeddings are concatenated to the input token embeddings. Only one soft prompt is learned, and it is universal for all domains and tasks. An illustration is shown in Figure 1. Code 1 shows the code for computing the SPC loss.

heading: 4 Experiments
text: We design experiments to validate the main claims of our SPC framework: (1) SPC loss can be plugged into any model or tuning method, and (2) improve the zero/few-shot in/cross-domain generalization. We will explain our selected tasks, data, model, implementation, and results.

heading: 4.1 Task description
text: We select standard text generation and classification tasks: (1) contextual query rewriting (CQR), (2) abstractive summarization, and (3) paraphrase detection.
CQR refers to rewriting a contextual query in dialogue into a non-contextual query such that any missing information from the context is completed. CQR plays a crucial role in conversation systems, and the rewritten query can be used for downstream tasks such as question answering, information retrieval, and reading comprehension. CQR can be addressed by rule-based approaches, retrievalbased approaches (Fan et al., 2021), pointer-based models (Rastogi et al., 2019; Elgohary et al., 2019; Quan et al., 2019), generation-based models (Anantha et al., 2021; Yu et al., 2020; Liu et al., 2021a; Regan et al., 2019). We combine the end-toend generation-based approach with prompt tuning to exploit both. Yilmaz and Toraman (2021) have attempted a soft prompt tuning approach for query rewriting. Abstractive summarization refers to generating a summary of a given text. Unlike extractive summarization, which extracts phrases from the original text, abstractive sum-
marization may generate new phrases (Lin and Ng, 2019). The difficulty is to balance the semantic cohesion between the text and the new generation. Such problem is addressed by attention methods (Narayan et al., 2018; Gehrmann et al., 2018), extraction-abstraction-based methods (Chen and Bansal, 2018), pointer-generatorbased models (See et al., 2017), generation-based methods (Zhang et al., 2020). Only a few use a prompt tuning approach (Li and Liang, 2021; Liu et al., 2022). Paraphrase detection refers to telling if two sentences mean the same (Yin and Schütze, 2015). Prompt tuning approaches have proven effective for this task (Lester et al., 2021; Gu et al., 2022).
Zero/few-shot cross-domain settings We train with all training data for in-domain tasks and evaluate on the test data. For cross-domain experiments, we adopt a few-shot one-to-one domain-transfer setting: first train with the source domain training data and then train with a single batch (16 samples in our settings) from the target domain training data. No target domain validation data is used. For the summarization task, we select five domains: sport, business, politics, entertainment, and technology. For the paraphrase detection task, we evaluate the few-shot cross-domain transfer between QQP and MRPC in both directions.
Zero/few-shot cross-task settings We conduct zero/few-shot cross-task transfer evaluation between CQR and summarization: take the trained model from the CQR task, evaluate on the summarization task, and vice-versa.

heading: 4.2 Data and metrics
text: The QReCC dataset for the CQR task consists of 14K open-domain dialogues with 80K (query, rewrite, answer) triples (Anantha et al., 2021). The XSum dataset for the abstractive summarization task consists of 227K (document, summary) pairs. The documents are collected from BBC news articles covering topics like sport, business, politics, entertainment, and technology. The summaries are of one-sentence length (Narayan et al., 2018). For both tasks, the original data split is used. Rouge score (Lin, 2004) is used as a standard evaluation metric.
Following Lester et al. (2021), we use the QQP (Shankar et al., 2017) and MRPC (Dolan and Brockett, 2005) dataset from the GLUE (Wang
et al., 2019) benchmark for the paraphrase detection task. QQP consists of 795K question pairs from Quora. Since there is no label for QQP’s test set, we use the validation set for testing and use 10% of the training set for validation. MRPC consists of 5.8K sentence pairs from online news articles. The original data split is used. Following Lester et al. (2021) and Raffel et al. (2020), we format the QQP question pairs as [question 1: <question 1 text> question 2: <question 2 text>]. For MRPC, we format the sentence pairs as [sentence 1: <sentence 1 text> sentence 2: <sentence 2 text>]. We convert the binary class labels (1, 0) to “same” and “different” to better situate the task in a generative setting. Only exact matches with the label word are considered correct. 1 F1 score is used as a standard metric.

heading: 4.3 Model and baselines
text: We test the utility of the SPC loss by plugging it into fine-tuning and prompt tuning approaches and comparing the before/after performance. Admittedly, many prompt tuning work surfaces at the time of this manuscript (Liu et al., 2021c; Chen et al., 2022b; Gu et al., 2022). However, it is not our intention to build a better mousetrap but to present a new problem—that of learning domainadaptable soft prompts—that naturally leads to a new learning paradigm. With this consideration, we select fine-tuning, PrefixTuning (Li and Liang, 2021), and (naive) prompt tuning (Lester et al., 2021) as our baselines because they are SOTA for respective tasks, most widely compared, and do not involve any additional components/tricks. Comparing these clean baselines present a clearer picture of SPC’s benefits. The baselines are:
(1) Fine-tuning the BART model (Lewis et al., 2020).
(2) PrefixTuning (Li and Liang, 2021), a strong baseline commonly compared by other work. It uses the pretrained BART model as the backbone and prepends prompt embeddings to the encoder and decoder layers. The prefix embeddings are reparameterized by feedforward neural networks (see Figure 1). We also implement a variant of PrefixTuning—only
1During inference, the generation length is 2 (including a start token). We observe that the T5 model can generate the exact “same/different” tokens with rare exceptions, so no words in vocabulary are masked off.
prepending prompt embeddings at the decoder layers.
(3) (Naive) prompt tuning (Lester et al., 2021), the simplest soft prompt tuning. It prepends prompt embeddings to the input layer (not all layers, see Figure 1). We use the same BART backbone model for CQR and summarization tasks and add prompt embeddings to decoder input. For the paraphrase detection task, we use the same pretrained and LM-adapted T5 models (Lester et al., 2021; Raffel et al., 2020) and add prompt embeddings to encoder input.

heading: 4.4 Training and implementation
text: CQR, summarization We use the pretrained BART-large model (Lewis et al., 2019) for finetuning. For PrefixTuning and SPC, we freeze the BART model and only tune the prompt-related parameters. PrefixTuning requires training prefix embeddings and the reparameterization networks. SPC requires training prefix embeddings and the weights in the loss equation (7).
Paraphrase detection The pretrained and LMadapted T5 models are used (Lester et al., 2021; Raffel et al., 2020). The T5-small model has 77M parameters, and the T5-XL model has 3B parameters.
All training and evaluation use comparable configurations and hyperparameters (e.g., batch size, learning rate, training epochs, prompt length, generation length, beam size). We find the optimal set of hyperparameters through searching, as shown in Table 8. The implementation is based on the HuggingFace’s Transformers library (Wolf et al., 2020). We also re-implement the PrefixTuning model (Li and Liang, 2021). We use the metrics code that comes with the HuggingFace datasets. Training is done on the AWS cluster with eight NVIDIA V100 or A100 GPUs. More details on hyper-parameters and computation costs are shown in Appendix A.

heading: 4.5 In-domain results
text: CQR Table 2 shows the results on the CQR task using fine-tuning, PrefixTuning, and prompt tuning with/without SPC loss. For each prompt tuning approach, we compare random prompt initialization (rand) and BART word embedding initialization (bart). Results show that the SPC loss plugin (row 8–12) uniformly increases performance over finetuning, PrefixTuning, and prompt tuning (row 1–7),
with the gain ranging from 0.6%–8.5%. For PrefixTuning (row 2–5), prompting only at the decoder layers (row 5) is the best among its variations. SPC loss also significantly reduces sensitivity to prompt initialization.
Summarization Table 3 shows the results on the summarization task. We observe similar trends to CQR. 2 The SPC loss increases performance over fine-tuning, PrefixTuning, and prompt tuning, with the gain ranging from 0.2%–7.5%. SPC loss also significantly reduces sensitivity to prompt initialization.

heading: 4.6 Cross-domain results
text: Given the performance boost of SPC on top of other tuning approaches demonstrated in the last section, we choose to use the simplest and most
2Our implementation does not achieve the performance reported in (Li and Liang, 2021). This may arise from different experiment configurations: compared to (Li and Liang, 2021), we do not use BPE tokens and length normalization, and we use smaller batch sizes and only 10% validation data.
parameter-efficient SPC loss + prompt tuning (SPC+Prompt) to demonstrate SPC’s utility in cross-domain settings.
Summarization Table 4 presents the results for both in-domain and few-shot (16) cross-domain summarization tasks obtained with fine-tuning, PrefixTuning at the decoder with BART initialization, and SPC+Prompt with BART initialization. For indomain, fine-tuning outperforms the prompt tuning methods. This aligns with our expectations, given that fine-tuning tunes the entire model, thereby utilizing a significantly larger number of trainable parameters compared to prompt tuning approaches. Remarkably, despite having far fewer trainable parameters, SPC+Prompt manages to achieve performance that is either comparable to or better than that of PrefixTuning. In some domains, it even matches the performance of fine-tuning. Furthermore, when plugged into fine-tuning, SPC has the potential to enhance its performance even further (as shown in Table 3). Most impressively, despite less sufficient in-domain training, SPC+Prompt surpasses other state-of-the-art methods in crossdomain performance. This greatly improves crossdomain adaptation.
Paraphrase detection Table 5 shows the results on the paraphrase detection task for both in-domain and few(16)-shot cross-domain between QQP and MRPC. SPC+Prompt initializes prompt embeddings with the first prompt-length token embeddings of the T5 model. SPC+Prompt outperforms prompt tuning with large margins, validating the cross-domain generalization of the soft prompts learned by SPC.
Cross-task Table 6 shows the results for zero/few-shot cross-task experiments between CQR and summarization. SPC+Prompt is at par or better in all settings. Prompt tuning approaches perform better than fine-tuning in low-resource settings, consistent with the findings by Chen et al. (2022a).

heading: 4.7 Analysis and discussion
text: Number of parameters PrefixTuning has around 7% trainable parameters (including prefix embeddings and reparameterization networks) compared to fine-tuning. Our SPC method drastically reduces the number of trainable parameters by using 0.07% parameters (including prompt embeddings and SPC loss parameters) as shown in Table 7. This ratio
becomes negligible as the LM goes larger (e.g., T5XL, GPT-3). We do not show SPC results with reparameterization as its effect is inconsistent across tasks (Liu et al., 2021c).
Ablation study Table 2, 3, and Figure 2 show the effectiveness of our SPC framework in comparison with baselines. The SPC loss significantly improves the performance of fine-tuning, Prefix-
Tuning, and prompt tuning.
Prompt length Prompt length significantly affects the performance of prompt tuning approaches (Li and Liang, 2021; Liu et al., 2021c), as shown in Figure 3. Their relationship is nonlinear. A longer prompt (hence more parameters) does not translate to better performance, suggesting prompt tuning’s high variance that cannot be stabilized by using more parameters or larger models (Chen et al., 2022a). Generally, simpler tasks require a shorter prompt (Liu et al., 2021c).
Prompt initialization Prompt tuning methods generally benefit from “meaningful” prompt initialization, such as initializing from model word embeddings as shown in Table 2 and 3. Our SPC approach is less sensitive to the initialization method than PrefixTuning (Chen et al., 2022a), and the SPC loss helps stabilize performance.
Prompt position Our experiments suggest that generation tasks favor prompting after the original input, i.e., at the decoder. For the text classification
task (paraphrase detection), we prompt before the original input as suggested by Liu et al. (2021c); Lester et al. (2021). However, in our experience prompting after the original input still performs better for most tasks.
Interpretability There is an ongoing debate on whether soft prompts are interpretable. Webson and Pavlick (2021) find little evidence supporting language models understanding the prompts’ meaning. Other works suggest that soft prompts form semantically close clusters (Lester et al., 2021; Zhang et al., 2021) and task-specific clusters (Su et al., 2022). However, the prompted embedding space may be highly nonlinear and potentially non-metrizable; hence, the clusters induced by Euclidean or cosine distance may not imply semantic similarity (Su et al., 2022). Instead, the clusters may be illusions arising from certain initialization strategies or the clustering tendency in high dimensions—namely, the probability for highdimensional vectors to be uniformly distributed across dimensions is low. Our experiments find that the learned soft prompts do not convert to meaning-
ful phrases. However, carefully controlled experiments are required to exclude confounding factors, and we leave this exploration to future work.

heading: 5 Conclusions
text: This paper proposes a soft prompt construction (SPC) framework to learn domain-adaptable soft prompts. The derived SPC loss is model/taskagnostic and can plug into other models and tuning methods to improve their cross-domain performance. Experiments on three tasks (contextual query rewriting, abstractive summarization, paraphrase detection) demonstrate that SPC improves significantly over strong SOTA: PrefixTuning (Li and Liang, 2021), prompt tuning (Lester et al., 2021), and fine-tuning in zero/few-shot crossdomain/task settings across different LMs (BART, T5 family). Various properties of SPC are also analyzed. This study paves the way for loss-guided domain-adaptable prompt learning.
Limitations
The proposed SPC framework is model and task agnostic and can scale to different models and tasks. It is adaptable to domain/task shifts and generalizes well with low data. However, the prompt length has to be tuned for each task, and there is no principled way of determining the optimal prompt length. This work does not explore the interpretation of soft prompts or prompt ensemble. Future work can investigate other loss forms derived from our SPC framework, explore the combination of fine-tuning’s strong in-domain performance and SPC’s cross-domain performance, and apply SPC and other parameter-efficient tuning approaches to large LMs.
Ethical Statement
We hereby state that our study adheres to the ACL Code of Ethics and Professional Conduct. We do not see an immediate negative impact or potential misuse of the proposed SPC method. Should our method fail in real-world applications, it will not cause potential harm to vulnerable populations. We examine the dataset and trained models and do not see bias incurred by our method. Our study does not use demographic or identity characteristics from users. No data is or will be collected from users. Our method does not increase energy and carbon costs but will potentially reduce them due to its data- and parameter-efficient training.

heading: A Additional Experiment Details
text: 1 # Using PyTorch as demonstration 2 freeze_params(model) 3 model.train() 4 5 # Initialize prompt embedding 6 prompt_embedder = torch.nn.Embedding( prompt_length , embedding_dim) 7 prompt_embedder.weight = torch.nn.
Parameter(model.shared.weight [: prompt_length , :]. clone().detach ())
8 prompt_embedding = prompt_embedder(range (prompt_length)) 9
10 # Add prompt embedding to model input and run forward pass 11 inputs_embeds = torch.cat(( prompt_embedding.clone(), input_embeds), dim=1) 12 outputs = model.forward(input_embeds) 13 14 # Get embeddings 15 E_x = outputs.encoder_hidden_states [0]
# shape (batch_size , sequence_length , embedding_dim)
16 E_y = outputs.decoder_hidden_states [-1] 17 E_p = outputs.decoder_hidden_states [0]
or outputs.encoder_hidden_states [0][: , <prompt indices >, :]
18 19 # Transform embeddings 20 E_x = W_x(E_x).squeeze (-1) 21 E_y = W_y(E_y).squeeze (-1) 22 E_xy = torch.cat((E_x , E_y), dim=1) 23 E_xy = W(E_xy) 24 E_p = W_p(E_p).squeeze (-1) 25 26 # Compute loss 27 spc_loss = (E_xy - E_p).pow(2).sum(dim =1).sqrt().mean() 28 loss = outputs.loss + spc_loss 29 loss.backward ()
Code 1: Pseudo-code for SPC loss.
Table 8 shows the hyperparameter search space. The prompt lengths we used for CQR, summarization, and paraphrase detection are 200, 200, and 120. To conduct a fair comparison, we use the same batch size and number of epochs for all model variants on the same task, regardless of tuning approaches. The (batch size, training epochs) for CQR, summarization, and paraphrase detection tasks are: (16, 50), (16, 50), (16, 200).
Fine-tuning tends to converge faster than parameter-efficient approaches. For example, finetuning (with/without SPC) reaches high validation performance with fewer epochs and slowly improves after that. Parameter-efficient approaches (PrefixTuning, prompt tuning, with/without SPC) take more epochs to reach the same performance. However, we do not use early stopping and use the
best checkpoint for evaluation after finishing all epochs.
The exact number of epochs for stabilizing performance varies with each setting and task. However, we can compare their effective FLOP (= number of trainable parameters × number of epochs to stabilization). For example, for the CQR task, the effective FLOP for fine-tuning, PrefixTuning, and SPC+Prompt are: 406M params × 20 epochs = 8120M FLOP, 41M params × 40 epochs = 1640M FLOP, 0.3M params × 45 epochs = 13.5M FLOP. Hence, even with more epochs, SPC+Prompt has a drastically smaller effective FLOP.

heading: B Methodology: Algebraic Perspective
text: The soft prompt tuning problem aims to generate the target conditioned on the prompted input, which is achieved by simply tuning the prompt embeddings rather than designing the prompt tokens. We aim to improve soft prompts’ cross-domain generalizability by proposing a domain-adaptable learning paradigm.
We examine the three spaces that play a crucial role here: general task (or input) space, prompted space, and specific task (or output) space. Denote the tuple (input space, prompted space, output space) as the three-spaces. The input space is the general task space because it contains the task information without referring to a specific task or target. For example, an LM cannot tell if text input is for rewriting or summarization without fine-tuning on the specific task (see Table 1). However, prompts in the prompted space provide the context to specify the task and steer the LM to predict targets in the specific task space (Reynolds and McDonell, 2021). For instance, Table 1 shows how a prompted input, which combines input and prompt, contains both the original text and task-specific information. Nonetheless, the hand-designed prompts are not
scalable to other tasks as they require manual labor, and traditional soft prompt tuning approaches do not generalize to new domains.
So how do we learn domain-generalizable soft prompts? We take an algebraic approach and resort to the category theory. Category theory is fundamental to other (modern) math streams and applied to other fields, such as computer science. It studies the universal properties of mathematical objects, their structures, and the relations between them (Barr and Wells, 1990). Using category theory, we can derive the universal properties of the three-spaces and their relations, and such properties will be natural to domains, i.e., they will be domain-invariant.
First, let us formalize the three-spaces into categories. The input space, prompted space, and output space carry the canonical structure of sets, so they are categories of sets (Barr and Wells, 1990): Seti, Setp, Seto. Within the category, the objects are sets of tokens (i.e., tokens of inputs, prompts, and outputs), and the morphisms are functions between sets. We introduce the functor: structure-preserving map between categories F : Seti → Seto, such that for any object x ∈ Seti, F (x) ∈ Seto, and for any morphism f : x → y ∈ Seti, F (f) : F (x) → F (y) ∈ Seto. Similarly, we introduce functors G : Setp → Seto and H : Seti → Setp. A commutative diagram 4 shows such relations.
Next, we push the category of sets to the category of vector spaces—that is, we move to the corresponding embedding spaces of the three-spaces. Embedding spaces naturally carry the structure of vector spaces, so they are categories of vector spaces: Vecti, Vectp, Vecto (they are actually categories of topological vector spaces as the embeddings carry manifold structure, but we omit this structure for simplicity) (Lang et al., 2002). For notional convenience, we omit the underlying field of real numbers for the vector space. The objects in the Vect category are vector spaces, and the morphisms are linear maps between vector spaces. Similarly, we have functors F̄ : Vecti → Vecto, Ḡ : Vectp → Vecto, H̄ : Vecti → Vectp. The map from the category of sets to the category of vector spaces is also a functor: the “embedding” functor.
To make Vectp domain-invariant, we must find invariance between functorial maps. We resort to homology: algebraic objects that are topolog-
ically invariant (Weibel, 1995). Denote the category of chain complexes over Vect as ∂Vect whose objects are chain complexes in Vect and morphisms are chain maps. There is a homology functor (Spanier, 1989): ◦ H : ∂Vect → Vect that maps from chain complexes to their homology (modules). However, computing the homology is non-trivial. To simplify the problem, we introduce a short exact sequence—simplest form of chain complexes and can be used to construct longer chain complexes (Massey, 2019; Weibel, 1995)— in Vect: K ≡ 0 → Ei h→ Ep g→ Eo → 0 where Ei, Ep, Eo are the input, prompted, output embedding spaces, and 0 is the zero object (exercise for readers to check such exact sequence exists). This indicates K’s homology
◦ H(K) ≡ ker(g)im(h) = 0id is
trivial and Ep is acyclic and invariant. Further, there exists a morphism h−1 : Ep → Ei such that h−1 ◦ h = idEi . By the splitting lemma (Weibel, 1995), we have Ep ∼= Ei ⊕ Eo. Hence, we can construct the prompted embedding space Ep from the direct sum of input embedding space Ei and output embedding space Eo, hence the name “soft prompt construction” (SPC). There are more than one approaches to enforce such a domain-invariant property. We choose the most straightforward one: the SPC loss (7).

heading: Future Work
text: Future work can investigate other loss forms derived from our SPC framework, explore the combination of fine-tuning’s strong in-domain performance and SPC’s cross-domain performance, and apply SPC and other parameter-efficient tuning approaches to large LMs