Stochastic Bridges as Effective Regularizers for Parameter-Efficient Tuning
Authors: Weize Chen
Xu Han
Yankai Lin
Zhiyuan Liu
Maosong Sun
Jie Zhou
Venue: Submitted to ICLR 2023 (2023)
Keywords: parameter-efficient tuning
pre-trained model
stochastic process

Abstract: Parameter-efficient tuning methods (PETs) have achieved promising results in tuning large pre-trained language models (PLMs). By formalizing frozen PLMs and additional tunable parameters as systems and controls respectively, PETs can be theoretically grounded to optimal control and further viewed as optimizing terminal cost and running cost in the optimal control literature. Despite the elegance of this theoretical grounding, in practice, existing PETs often ignore the running cost and only optimize the terminal cost, i.e., focus on optimizing the loss function of the output state, regardless of the running cost that depends on the intermediate states. Since it is non-trivial to directly model the intermediate states and design a running cost function, we propose to use latent stochastic bridges to regularize the intermediate states and serve as the running cost of PETs. As the first work to propose regularized PETs that use stochastic bridges as the regularizers (running costs) for intermediate states, we show the effectiveness and generality of this regularization across different tasks, PLMs and PETs. In view of the great potential and capacity, we believe more sophisticated regularizers can be designed for PETs and better performance can be achieved in the future.

Under review as a conference paper at ICLR 2023
STOCHASTIC BRIDGES AS EFFECTIVE REGULARIZERS FOR PARAMETER- EFFICIENT TUNING
Anonymous authors Paper under double-blind review
ABSTRACT
Parameter-efficien tuning methods (PET's) have achieved promising results in tun- ing large pre-trained language models (PLMs). By formalizing frozen PLMs and additional tunable parameters as systems and controls respectively, PETs can be theoretically grounded to optimal control and further viewed as optimizing the ter- minal cost and running cost in the optimal control literature. Despite the elegance this theoretical grounding, in practice, existing PETs often ignore the running cost and only optimize the terminal cost, i.e., focus optimizing the loss function of the output state, regardless of the running cost that depends on the intermediate states. Since it is non-trivial to directly model the intermediate states and design a running cost function, we propose to use latent stochastic bridges to regularize the intermediate states and the regularization as the running of PETs. As the first work to propose regularized PETs that use stochastic bridges as the regu- larizers (running costs) for the intermediate states, we show the effectiveness and generality of this regularization across different tasks, PLMs and PETs. In view of the great potential and capacity, we believe more sophisticated regularizers can be designed for PETs and better performance can be achieved in the future.
INTRODUCTION
Recent years have witnessed the dramatic growth of pre-trained language models (PLMs) in various fields (Devlin al.] 2019] Dosovitskiy et al.) 2021). As the size of PLMs continues increase, the number of parameters has now even reached hundreds of billions (Brown et al., 2020 Smith al. 2022), making fine-tuning the whole PLM both computationally impractical and environmen- tally unfriendly. In view of this, variety of Parameter-Eificient Tuning methods (PETs) are pro-
posed (Houlsby et al. 2019:; Hu et al. 2022 Zaken et al. 2022 Lester et al. 2021). By only tuning small number of additional parameters, PETs can be comparable to full-parameter fine-tuning.
Despite the success of PETs, their underlying mechanism remains an open problem. Recently, several works have proposed interpret PETs with optimal control theory. Yang & Liu (2022 first show that the optimization in Prefix Tuning (Li & Liang 202I) (a typical method of PETs) can be considered the search for optimal control variables in the context of optimal control, i.e., the trainable prefixes can be seen as the control variables that drive the PL.M (the system) to the desired output. Diig et al. (2022) further show that the optimal control perspective can be applied to almost all PETs. The optimization of PETs' parameters can be seen as minimizing the two cost functions in the optimal control literature: (1) terminal cost Lr, which measures the quality of the terminal state, and (2) rumning cost LR. which measures the feasibility of the controlled intermediate states and the control variables. Although Lr can well correspond to the loss function of the model output. L is only vaguely described as the regularizers on the parameters of PETs (control variables) in Yang & Liu 2022) and [Din et al. (2022). ignoring the dependency of Ln on the intermediate states.
In this work, we show that designing a running cost to regularize intermediate states not only makes the optimal control perspective of PETs more theoretically sound, but also empirically leads to better PETs. We begin by assuming that in PLMs, the intermediate hidden states for generating different tokens in a sentence have different dynamics (or trajectories). and the dynamics can be approximated with stochastic processes in latent space. Specifically, first freeze the PLM and learn mapping from the original hidden state space of the PLM to latent space. the latent space, the dynamics of the intermediate hidden states for generating different target tokens be approximated with

---

Under review as a conference paper at ICLR 2023
Running cost: Ex
Teroioal cosk Cy
Endpaitt Oetorinaaionn
Maduue
Thal [ss
Tall [oute]
PCA
Layee
Mossing Approsination Approimaatng Rrasltan PDF
Adagne
Approslrating
Pretraining
Ik Kg)
1 GLUE
Tau bu (x) bnldg
downebream
Frompt
Trandioon PDP
Figure 1: An overview of our proposed latent stochastic bridge regularizer.
different target-specific diffusion bridges. The obtained mapping can then be plugged to the model regularize the intermediate hidden states when training PETs parameters. Besides, since diffu- sion bridge is (1) Markov process and (2) a solution to stochastic differential equation (SDE), we correspondingly propose two methods to learn the mapping: (1) fitting the Markov transition prob- ability density function (PDF) and (2) fitting the SDE directly. These two methods act as trade-off between efficiency and effectiveness: the first method incurs only negligible computational cost and has satisfactory results, while the second one is slower, but yield better regularizers.
We conduct experiments on different PLMs of different sizes, and the experimental results on GLUE (Wang et al. 2019) under both full-set and few-shot settings demonstrate the effectiveness of our proposal across four different PETs Further analyses show that the learned regularizer helps pull apart the hidden states of different label words. We also observe that when project the in- termediate hidden states of PETs without our regularizer into our latent space, the better the PETs latent bridges. This spontaneous approaching behavior
perform, the closer the latent states may indicate that stochastic- -eriaattti--biie--i--- latent dynamics naturally exists in well-trained PETs.
In summary, our work has the following contributions: (1) Guided by the perspective of optimal control for PETs, we design latent stochastic bridge regularizers on the intermediate states during the training of PETs. (2) We propose two methods construct the latent space according to the two representations of stochastic bridges, offering a trade-off between efficiency and effectiveness. (3) Our regularizers are shown to be effective and general across different PLMs, different PETs, and different tasks. (4) We show that well-trained PET's without any regularization spontaneously exhibit boocasttc--ridge--le latent dynamics.
BACKGROUND
2.1 DEFINITION AND MATHEMATICAL. NOTATIONS
Consider using a L-layer PLM with the vocabulary to handle a text-to-text task D. For each sample (æ, y) E D, y E V is the output token and VN is the input token sequence where N is the length of æ. With æ as the input, each layer of the PLM will output a sequence of hidden states,
and we denote the hidden states of the i-th PLM layer ()) (h) (0) N 1==1, where h.)) RR is the state at the position j of the i-th layer, and d is the model dimension. We denote the position where the model outputs the target y as o, i.e., the model should predict y with the hidden states (T))
2.2 OPTIMAL CONTROL PERSPECTIVE OF PETs
Conventionally, adapting the PLM to D requires full pallmparretter fine-tuning, which is given as: min Ez, DDD ( v) R(A0).. h { h (i-1) + gffi oo (h (i-),, i= 1,...,,4 L, Embed(w))
(1)
0,
Herre we assume y V since sample where E V can he decomposed to M samples, The i-th sample (Iæ: I. ya) for auto-regressive language modeling or ([æ; y-il. y1) for auto-encoding language modeling.
2

---

Under review as a conference paper at ICLR 2023
where 0 is the PLM parameters, A0 is the full-parameter update, [(:-.) is the loss function, R(-)) is the regularization function, ( (-) is the forward propagation of the i-th PLM layer after updating the parameters, Embed transforms the input token sequence into input embeddings.
As |0] continues to increase, full parameere fine-tuning becomes impractical, and various PETs are proposed to mitigate this problem. Let Φ = [/ be PETs' parameters. Ding et al. (2022) give unified view of PETs from the perspective of optimal control, and Eq. can be re-written
h ((-1) + 96 (n )("), i= i= L, 1 .0) + [ h = 10 Embod(x))
min F Cr (r
i= 0,
where g00 (:. ) represents the forward propagation of the i-th PLM layer intervened by PETs, [: is the concatenation operation, Cr(,,)) is the terminal cost and CR(-) is the running cost. Since |6| < 10). PETs can greatly reduce the tuning cost (more details in Appendix A
Typically for classification and generation tasks, Cr corresponds the cross-entropy loss of the prediction, and Ln can be seen as a regularizer on PETs' parameters é. However, in the optimal control literature, Ln depends on not only the control variables Φ, but also the controlled intermedi- ate states {h
( } In this paper, we show that including intermediate states to build L (i.c., using
E'zo CR ( (6(0), hh!") instead of Σ CR (o(()) not only makes the optimal control perspective of PETs more theoretically sound, but also empirically leads to better PETs.
2.3 DIFFUSION BRIDGES
diffusion process (Xe)rr) continuous-time Markov process. For any la < lbx the diffusion process equipped with a transition Probability Density Function (PDF) p(ts, ta, a), which gives the probability density of reaching b at time l given the history of reaching a at time fa- A diffusion process is also the solution to an Itô SDE dXX, put, X,)dt + at. X,,d,,, where B, a standard Brownian motion, (u:s)) is called drift function and (:.)) is called diffusion function.
A diffusion bridge XT; is a diffusson process conditioning on the path observations of the two endpoints (0, a) and (T" B), i.e., XTTa, in this work, and omit the superscript a. We consider two typical diffusion bridges, the Brown- bridge (OU bridge). We present here the properties of the
and X XT B. For simplicity, assume 0
ian bridge and the Brownian bridge and leave the properties of OU bridge to Appendix B
Proposition 2.1 (The properties of the Brownian Bridge). A Brownian bridge T: with Ti8 is the solution the following SDE:
and X i
dX. (8 XT) 1) dt + dB,, Xo
(3)
The transition PDF from x XT 0 to T to
is given as Ts (to, b 0,0) (2mtu(T-1s) exp
(h- (t/ 24, (T' -1))
Diffusion bridges and SDEs are battle-tested tools to describe the stochastic dynamics of complex systems in engineering (Sobczyk 2013). finance (Wang & Sloan, 2011). biology (Horsthemke & Lelever 1984). etc. Considering the dynamics of PLMs hidden states are necessarily complex, diffusion bridges and SDEs serve as ideal tools for us to model the dynamics.
3 LATENT STOCHASTIC BRIDGES REGULARIZER
3.1 THE OVERALL FRAMEWORK
Building latent dynamics in the latent space. Since directly regularizing the intermediate states and constructing the running cost are non-trivial. we introduce projection from the intermediate state space to a latent space, and leverage diffusion bridges as regularizers to construct the running cost. Specifically, we define a r-dimensional latent space U C R'(r < d) and a learnable mapping
0y iRd x Rd U, where 7 denotes the parameters. I- projects the hidden state h. and its context
3

---

Under review as a conference paper at ICLR 2023
state (+) into the latent space U at each layer of the PLM. Since he is contextualized while latent bridges are not, introducing the dependency on h()) can inform 9, about the context at the i-th layer and allow g-y to decontextualize the hidden states. We simply take the averaged states the i-th
layer h (1) N ΣØ h as the context. We define the latent states with discrete time in U as
(5) where 1,41 is the normalized layer index. We include the 0-th layer (input layer) because some PETs (e.g., prompt tuning) act on the 0-th layer. We use to 0, f2 = I represent the two endpoints.
(D (Lo) = (f++1.9 ) f+ = (+ 1)/(1) 22,,
By using natural cubic spline knotted { =i= to interpolate over [-1, T. + 11, we further give
continuous representation of the states in the latent space U as
(6) Learning the mapping from hidden state space to latent space. Since adapting PLMs to down- stream tasks can be seen as transferring the knowledge obtained from pre-training tasks to down- stream tasks, we argue that the latent dynamics of intermediate hidden states for generating the same token y should be similar in both the pre-training and downstream tasks. Therefore, we train the mapping J, on the corpus that is used to pre-train the backbone PLM, and then apply the learned mapping to downstream tasks to encourage the latent dynamics to be similar to that in pre-training.
uc(o,, (h( )xE)-1,441)) (f2+11
(h
(h( (*)) )=E()1,
f2+1 (ar + 1)/(1 + 2) E [0, 1].
Specifically, we assume that the states to generate the token in the latent space U form trajectory that is path sampled from X with high probability, where X 1i8 is the pre-determined diffusion bridge describing the latent dynamics to generate y. and 3, is the tail endpoint of the diffusion bridge. More details of X 1:99, will be discussed in Section 3.2
(0) On the corpus where the PLM pre-trained, we fix the PLM and use its hidden states (h \==1 to learn 9, by maximizing the goodness of approximation for latent states u under the bridge X1:
7 arg max [goodness-of-approx ((rg (1,))))
xl1xx)]
where u can be up (Eq. 5) uc (Eq. 6 depending on the fitting method,
also a function depends on the choice of the fitting method, measuring how likely u is a sample trajectory of X 1.8, In Section 3.3 we will define this function alongside the fitting methods.
Regularizing PETs with latent dynamics. After learning 9-, with Eq. 7 we freeze y and use the offtiness-of-pppreee function as the running cost in Eq. for PETs on downstream tasks. The final objective for PETs becomes
C= Cr vv) + a gf (h! 1), x118)),
(8)
where the second term is the running cost and is a hyper-parameter controlling the regularization strength. By optimizing Eq. 8 PET's need to ensure the final hidden state is capable of predicting y, while keeping the latent states at the position conform to the diffusion bridge X1:8, Note that introducing I- as the regularizer does not increase the number of trainable parameters for PETs during the training stage since 7 is fixed. Also, 9y does not involve in the inference stage.
3.2 DETERMINING ENDPOINTS FOR DIFFUSION BRIDGES
An intuitive approach to determine the endpoints for the diffusion bridges for each target token is to optimize the endpoints together with the mapping 97. However, optimizing endpoints and 9. jointly may admit a trivial solution: endpoints are both 0 E TR" and 9- always outputs O. Since is always a point in the sample path of such degenerated diffusion bridge, the value of goodness- of-approximation function can be meaninglessly high. Although sophisticated constraints can be imposed here, as the first work that uses diffusion bridges as regularizers, we simply pre-determine the endpoints and keep them fixed, and remain introducing constraints as our future work.
Specifically, we apply principal component analysis (PCA) to the output token embedding matrix V R [V] xd of the PLM, obtaining r-dimensional embedding matrix, and re-normalize each row to have norm 17. Let the resulting embedding matrix be 3 E IR 1 We then use 0 E R2' as the heads for all the bridges, and B as the tails of the diffusion bridges, i.c., the y-dimensional embedding of y in (9 is used as B, in X 1,8, The intuition for using /3 as the tails is that the trajectories of the intermediate states for similar target tokens should be close. In V, similar tokens are close, and B obtained by PCA can well preserve the token similarity after reducing dimensions.
4

---

Under review as a conference paper at ICLR 2023
3.3 FITTING THE MAPPING 9y
We use the Brownian bridge to illustrate the fitting of Jy. It can be analogous to OU bridge easily. Method 1: Approximating the Transition PDE, Generalizing Eq. 4 high dimension, we can derive the transition PDF from (0, 0) to (l++1. 9r ( (()) for X
p buuy (1 9. (h h 0) 0, 0) x exp( 10-ths - 26441
L),
(9)
where 4 has the same definition as that in un (Bq. 5 To make Jy approximate the transition PDF, maximize the sum of log-probability of up under the Brownian bridge X X1:9;;
goodness-of-approx E'= log (p)
(1441, ghy (h. R 0, 0)] + const.
(10)
Here, 9y be expectation of the Brownian bridge X1:8% at {t++1)/=o-
mapping from the hidden state space the latent space by predicting the
Method 2: Approximating the SDE. Since the Brownian bridge is the solution the SDE in Eq 3
we can let gy approximate the SDE. Solving the SDE requires continuous latent states, while we only have L+ 1 discrete observations, we thus use the continuous representation te introduced in Eq. 6 Generalizing Eq. 3 to high dimension, the SDE approximated by 9n can be defined as:
d7, 9r (h( h (x) t)dt + dBe,
(L 2) 1,
(11)
where x: is the same as that in Eq. 6 B [0. - + R' is standard r-dimensional Brownian motion.
Here, we additionally introduce the dependency on for g., since time information shown to be important in previous neural differential equation works (Zhang et al. 2020 Dupont et al.) 2019). Following Li et al. 2020), when two SDEs share the same diltusion function, the KL divergence between the probability measures induced by the two SDEs is finite. Since the diffusion function
for Eq, and the multi-dimenssonal generalization of Eq, 3 the KL divergence between the probability measures Joy Eq. aad µ of generalized Eq. can be estimated by:
(RL (,av) = =BZ[ //" 2 [|aa(t,))
(12)
(z) (()) .) (ut, Ze)) o, (5) (1), B, Z. u(t, = o (
1-t
where ()(:)) is the drift function of the pre-determined Brownian bridge X1:9,, We use the KI. divergence the goodness-o--aprroiiaat seen as a mapping from the hidden state space to the latent state space by approximating the drift vector field of the underlying Brownian bridge liBy.
function optimize the mapping 9y, Here, I-, can be
4 EXPERIMENTS
To verify the effectiveness and generality of the regularizers built stochastic bridges, duct experiments on (1) different PLMs: BERT large (340M) (Devlin et al.] 2019) and Debertagtage (750M) (He et al. 202T): (2) different PETs: Prompt tuning, LoRA, BitFit and Adapter: (3) differ- ent diffusion bridges: Brownian bridge and OU bridge. We show that the regularizers effectively improve the performance GLUE (Wang al. 2019) under both full-set and few-shot settings.
4.1 EXPERIMENTAL SETUPS
Datasets. Since both BERT large and Debertastarge use Wikipedia and BookCorpus ZZhu et al. 2015) for pre-training, we thus use these two corpora to train 9-- We report accuracy for MNLI, SST-2, QNLI, and RTE; Fl for MRPC and QQP; and Matthews correlation for CoLA. We report the average performance and the standard deviation on the development set over different runs. We append a special token [MASK] to each sequence, and require the PLM to output the label word at [MASK] (e.g., negative or positive for SST-2). We exclude STS-B for it a regression task.
Models and PETs. We use the checkpoint released by Megatron (Shoeybi al. 2019) for BERT large and the official v1 checkpoint for Debertä We use a simple three-layer MLP to build gy- For Prompt tuning,
soft prompt of length 20, and append it to the end of each
5

---

Under review as a conference paper at ICLR 2023
Table 1: The results on GLUE for BERT large- The values are the average value of the best per- formances over three different runs, and the subscripts are the standard deviations. The A column
shows the difference of the average performance between the vanilla PETs regularized PETs.
PET
MNLI
QQP
QNLI
SST-2
MRPC
CoL.A
RTE
Average
A
PROMPT
84.4o.1
85.30.3
91.5a.1
95.50.1
73.92.4
55.53.4
60.81.5
78.10.6
+BROWN_P .DF
84.7a.2
85.5u.9
91.8u.6
95.7u.1
75.4u.a
56.4a.
61.52.2
78.7u.4
0.6
+BROWN SDE
84.90.2
85.4o.1
91.80.
95.80.3
78.8
61.42.9
64.7
80.40.2
2.3
LORA
88. 88.8p.1
89.2n.2
93.50.2
95.50.1
84.60.4
62.81.6
78.91.6
84.8p.3
ABROWN_PDE
88.9b.1
89.6u.1
93.9u.1
95.6u.2
85.10.7
63.7u.5
80.0p.s
85.2p.1
0.4
+BROWN.ShD SDE
88.9..1
89.5o.1
93.7p.1
95.70.1
86.5
63.90.4
80.90.8
85.60.1
0.8
BITFIT
87.90.2
87.6p.1
92.70.2
95.60.1
79,42.3
60.2o.s
77.01.a
82.9p.3
+BROWN PDF
87.90.1
87.8u.9
93.00.2
95.70.1
83.1o.s
60.30.6
78.3p.9
83.7p.2
0.8
+BROWN-SDD
87.9p.2
87.7u.a
92.8u.1
95.7u.1
83.3u.s
61.11.2
77.71.a
83.8p.a
0.9
ADAPTER
88.8p.
89.6p.3
93.70.1
95.6p.1
83.6n.1
60.41.2
79.51.2
84.5p.3
+BROWN PDF
89.0p.1
89.70.2
93.80.2
95.80.1
86.51.1
62.60.7
83.20.2
85.80.2
1.3
+BROWN_SDE 88.9b.1
89.8u.1
93.90.2
95.8u.1
85.9u.4
62.31.8
82.2g.2
85.5p.2
1.0
sequence. For LoRA, we apply to the query and value of attention modules. For Adapter, we apply it to the output of attention and feed-forward modules. For BitFit, we tune all the bias terms in linear layers and layer normalization modules. Hereafter, we use PDF regularizer to refer to using Ir fitted by approximating the transition PDF, and SDE regularizer to refer to using 9,, fitted by approximating the SDE, vanilla x' to refer to the PET x without using regularizers.
Few-shot Experiments. We randomly sample 2 k examples from the original training set Duain for each class. The sampling is performed times with different seeds to form 5 training sets and development sets (D train D(t) dev with each being k-shot. Each time we train PETs D(s)
irinn
select the best model on D(i) and report its performance on the original development set D dev:
Hlype--parameters We list syper-parameters in Appendix E We mainly focus the difference in performance between vanilla PETs and regularized PETs. Therefore, we directly set the hyper- parameters to reasonable values and do not perform much hyper-parameter search. But we ensure the hyper-parameters for vanilla PETs and regularized PETs are the same for a fair comparison.
4.2 FULL-SET RESULTS
The experimental results for BERT large are reported in Table Due to space limitation, see Table for the complete results including OU bridge regularizers. The first line of each block in the table
the performance of vanilla PETs, and the rest of the lines are the performances of the regularized PETs. We also conduct the same experiments for Debertaslage and place the results in Appendix C
In general, both Brownian and OU bridges, and both PDF and SDE regularizers are able to effec- tively improve the performance of PETs, showing the effectiveness of our proposed regularizers. Particularly, for Prompt tuning, the SDE regularizer with both diffusion bridges yield an increase on the average performance of more than 2 points. We assume that it is because Prompt tuning has far less trainable parameters than other PETs, and it only acts at the input layer, which is far from the supervision signals of the terminal cost Lr. Therefore, when provided with the regularization on the hidden states, the prompts receive more guidance and eventually reaching a better local optimal.
Overall, the two diffusion bridges in experiments do not show much difference. As for the two fitting methods, SDE regularizer is generally more effective, especially for Prompt tuning where the number of trainable parameters is restricted. However, we also observe that SDE regularizer about 3 times slower than PDF regularizer, which brings the trade-off between performance and efficiency. One can expect better performance by leveraging more sophisticated underlying stochastic bridges, exploring more reasonable endpoints for bridges and designing better mapping 9-. As the first work using latent stochastic bridges as regularizers, we mainly consider the most straightforward cases and aim to show the great potential of the approach and the importance of regularizing hidden states.
4.3 FEW-SHOT RESULTS
We observe that in Table [ the improvements brought by our regularizers are more substantial on small datasets MRPC, CoLA and RTE. We assume that this is because in rich-resource datasets,

---

Under review as a conference paper at ICLR 2023
Table 2: The results on GLUE for BERTlage under the 16-shot setting, We exclude CoLA because
all PETs fail to give reasonable results under the fhw-shot setting.
PET
MNLI
QQP
QNLI
SST-2
MRPC
RTE
Average
PROMPT
1..
53.0g.1
51.61.4
7..
50.13.0
48.0 48.01.3
51.80.9
+BROWN_PDD
38.72.3
54.92.8
52.11.1
75.011.0
52.82.
50.83.3
54.01.s
2.2
+BROWN SDE
40.60.8
55.42.1
52.91.6
80.010.9
51.93.6
51.73.1
55.41.5
3.6
LORA
48.74.5
59.9s.5
53.21.2
90.21.1
53.63.4
64.20.9
61.6p.6
+BROWN_PDF
52.01.3
62.72.2
55.
91.30.1
57.4s.0
65.31.8
64.0u.7
2.4
1BROWN_SDE
54.10.9
65.51.4
64.36.1
91.20.3
60.23.1
65.81.4
66.8p.a
5.2
BITFIT
48.41.6
56.0g.1
51.72.6
90.8u.s
52.02.3
61.71.4
60.11.1
+BROWN PDF
48.51.9
56.0g.u
53.52.0
91.0g.4
53.92.3
63.21.6
61.0p.a
0.9
+BROWN
52.30.5
61.22.9
58.84.7
90.8o.4
54.83.0
63.92.5
63.60.8
3.5
ADAPTER
47.43.7
57 57.07.2
55.82.9
91.0g.4
55.82.6
62.72.0
61.61.2
+BROWN
49.04.8
58.57.4
56.9a.1
91.40.2
57.24.9
63.23.0
62.71.a
1.1
+ +BROWN_SDE
52.32.2
62.42.9
64.81.5
91.90.4
57.34.1
63.81.6
65.41.3
3.8
the abundant data itself has provided enough information to learn high quality PETs, while in low- resource datasets, the data insufficient and the regularizer can offer additional helpful supervision. To validate this, we conduct the experiments under the few-shot setting GLUE.
The 16-shot results are shown in Table 2 and the results for the OU bridge, results for 4-, and 32-shot and results for Debertadaage are placed in Appendix C For all PETs, the SDE regularizer yields an improvement of more than 3 points. Particularly, the SDE regularizer on LoRA brings an improvement of 5.2 points. By applying regularizers under the few-shot setting. there is now sub- stantial boost on what was originally a rich-resource dataset, such as MNLI, QQP and QNLI. The PDF regularizer also gives modest improvements. Although slightly inferior to the SDE regularizer, is still satisfying, considering that the PDF regularizer brings such a performance improvement sig-
with little computational cost introduced. We additionally observe that the improvement nificant on Debertaxtage in Table demonstrating the potential of our regularizers on larger models.
5 ANALYSES
In order to have a better understanding of the role played by our regularizers, we analyze the hidden states of the PETs with and without regularizers in this section. We choose Prompt tuning as a representative in the analyses. By varying the hyper-parameter in Eq. g we show that as the strength of the regularization gets stronger, the clusters hidden states corresponding to different label tokens become compacter and more distinguishable. Also, show that the hidden states of vanilla PETs spontaneously approach the latent bridges in the latent space without knowing the bridges, indicating that there may exist intrinsically fnffusiion-brdge-llii latent dynamics for PETss
5.1 THE REGULARIZER WIDENS THE DISTANCES BETWEEN LABEL CLUSTERS
We use the different prompts obtained with or without regu- larizers on the full-set GLUE, and record the intermediate hid-
Table 3: Pearson's 's correlation be-
tween the regularization strength and the average distance be- tween the centroids of different la- bel clusters. ***; p < .001, **:
den states at the position of prediction as (i) [M We vary the regularization strength by adjusting the coefficient a in Eq. to inspect the impact of the regularization strength on the hidden states. Note that when a vanilla PET.
O, it degenerates to the
< .01, < .05.
Cocf.
We randomly sample 100 samples for each label in MNLI, and use UMAP (McInnes et al., (2188) to reduce the dimension of the last layer's hidden states of different prompts, and plot them in Figure 2 It shows clearly that for both regularizers, as the regularization strength becomes stronger, the hidden states of the last layer becomes more distinguishable among different labels. By looking at the axes these plots, we find that the distance between the clusters generally increases when the reg-
Dataset
PDF
SDE
MNLI
0.928
0.894***
QQP
0.715*
0.897***
QNLI
0.971**
0,837*
SST-2
0.966**
0.867**4
MRPC
0.226
0.433
CoLAA
0.633*
0.807
RTE
0.440
0.589*

---

Under review as a conference paper at ICLR 2023
LAM
(a) w/o regularizer
(b) SDE 0.001
(c) SDE a 0.01
(d) SDE 0.1
18
(e) PDF & = 0.01
(1) PDF a 0.1
(g) PDF a 1.0
Figure 2: A visualization of the last layer's hidden states MNLI using the prompt that is trained (a) without regularizer (b-d) with the SDE regularizer (e-g) with the PDF regularizer.
ularization strength is increased. We also notice that the SDE regularizer better helps separate the hidden states of the last layer by substantially enlarging the distance between the centroids of differ- ent labels, which could be of the reasons why the SDE regularizer has better effectiveness in almost all our experiments.
We also calculate the Pearson's correlation between the regularization strength and the average distance between the centroids of different clusters. The results are shown in Table 3 On all the datasets, the regularization strength has positive correlation to the average centroid distance, and on most of the datasets, the correlations are significant in the sense that the p-value < .05. This indicates that as the regularization strength becomes stronger, the centroids of different label clusters become more distant, which is a desired effect because the regularizer encourages the hidden states for different label tokens to conform different latent diffusion bridges.
5.2 HIDDEN STATES APPROACH SPONTANEOUSIA TO THE LATENT BRIDGES
An interesting phenomenon we observe is that the vanilla PETs' in- termediate hidden states spontaneously approach our latent bridges when they are projected by our mapping 9-,. That is. applying our mapping 9, to the hidden states of vanilla PETs, we find that when the performance of vanilla PET's becomes better, the average distance from 9- ((h ()) hOp to our latent bridge gets closer. Here, similar
Table 4: Kendall's rank cor- relation between the number of shots and the distance to the latent bridges.
Dataset
Coef.
p-value
MNLI
-0.39
.026*
Wang al. (2022). . we define the distance from 9- (h.) h)))) to its corresponding latent bridge X,, using Eq, 10 without the constant.
QQP
-0.36
.037"
QNLI
-0.32
.069
Note that the vanilla PETs have no access to 9y and the latent bridges during the training process, and gy also has no access to the PETs during its fitting process.
SST-2
-0.50
.005**
MRPC
-0.21
.255
RTE
-0.34
.049*
We demonstrate the above observation by conducting analyses in few-shot scenarios and reporting the correlation between (1) the number of shots and the average distance from latent hidden states
to latent bridges in Table 4 (2) the performance and the average distance from latent hidden states to latent bridges in Table 5 Specifically, we report Kendall's 's rank correlation for (I), and Pearson's correlation for (2). See Appendix D for the detailed setup for the calculation of the correlations.
Thhe that choose Kendall's rank correlation that it is suitable for data with ties. See Appendix
8

---

Under review as a conference paper at ICLR 2023
From Table 4 the number of shots has negative correlation to the distance, and the correlation is significant on 4 out datasets. This indicates that the amount of available data increases for vanilla PETs, its intermediate hidden states in latent space spontaneously approach latent bridges even without knowing the mapping 9, and the bridges. Additionally, the results in Table [ show the negative correlation between the performance of vanilla PETs and the distance to the latent bridges, and it is significant on 3 out of 6 datasets.
Table 5: Pearson's corre- lation between the perfor- mance and the distance to the latent bridges.
Dataset
Coef.
p-value
MNLI
-0.09
.715
QQP
-0.42
.062
Altogether, the two findings on correlation show that as the PETs learn to solve the problem. their intermediate hidden states sponta- neously approach our latent bridges in the latent space projected by Dr- This implies that there exists intrinsically ceffusion-bridge-lle latent dynamics for PET's and also justifies our use of latent diffusion bridges as regularizers.
QNLI
-0.77
<.001 **
SST-2
-0.69
<,001 *
MRPC
-0.13
.591
RTE
-0.87
<,001"
RELATED WORKS
Recent years have witnessed the success of PLMs (Raffel et al. 2020 Brown et al. 2020), which acquires rich knowledge from unlabeled data in a self-supervised manner and can be adapted to spe- cific tasks by tuning their parameters. Despite the success of PLMs, as the size of PLMs continues to grow, it becomes increasingly impractical to perform full- pall-eareettrr fine-tuning for downstream tasks. Therefore, various recent efforts have been devoted PETs, aiming to freeze PLMs and only tune few additional parameters for task adaptation, such as Prompt tuning (Lester et al. (02T) that prepends tunable tokens per task to the input text, Adapter (Houlsby et al., 2019) that inserts small modules into each layer, BitFit (Zaken et al.) 2022) that tunes only the bias terms, and LoRA (Hu) ct al. 2022) that decomposes the weight updates into low-rank matrices. In this paper, based on the theoretical grounding of PETs optimal control (Yang & Liu, 2022) Ding et al. 2022), we develop stochastic bridges as the regularizer for intermediate hidden states and introduce regularized PETs, showing effectiveness and generality on different PLMs and tasks.
Since we adopt latent stochastic bridges as the regularizer, our work closely relates to continuous- time neural ODEs (Chen et al.] 2018: Rubanova al. (019) and neural SDEs (Li al., [019)] Kidger et al. (022). Continuous-time neural ODEs and SDEs model the dynamics of the hidden states with ODEs and SDEs parameterized by neural network respectively For example Chen et al. 2018 show the resemblance between ODE and ResNet (He et al. 2016). and propose to learn the neural ODEs the generalization of ResNet in continuous time. Inspired by these works, we use SDEs to represent the latent dynamics of PETs in the latent space. Our work differs from these work in that we focus on using neural SDEs as regularizers for intermediate hidden states, rather than feature extractors on downstream tasks.
We also notice that Wang et al. (2022) explore the use of Brownian bridge in PLMs. However, they use Brownian bridge to regularize the text dynamics across time, while we use Brownian bridge to regularize the dynamics of intermediate hidden states across model's layers. We additionally show that diffusion bridges other than Brownian bridge can be easily applied in our regularizer. As far as we know, our work is the first to show the certusion-brigge-ee dynamics for hidden states across layers and use diffusion bridges the regularizer for intermediate hidden states.
CONCLUSION
In this work, we start from the optimal control perspective of PETs and we notice that the existing PETs lack a running cost that regularizes the intermediate hidden states. Therefore, we propose to use stochastic bridges in latent space as the regularizers for PETs. Experimental results on different models, different tasks and different PETs show that the proposed regularizer effectively improve the PETs' performance. Our analyses further show that when the PETs are trained with- out the regularizer, the hidden states spontaneously approach our diffusion bridges, indicating that there exists intrinsically tiffuusion-brige-- like dynamics for PETs. As the first work using stochastic bridges as regularizers, we show its effectiveness and generality even with simple diffusion bridges. We believe it is promising direction and we are excited to more future work.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Ties Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL 1457c0d6bfcch 9677777777 (ww.//44967778888888 www.4488o8114220.0
Abst html
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordi- nary differential equations. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 31: Annual Conference on Neural Information Pro- cessing Systems 2018, NeurIPS 2018, December 3-8, 2018, Moniréal, Canada, pp. 6572 6583, 2018. URL http::::s/ppccceeinns. neurips. hcc/paper/2c18/haae Btt.://........ee222242248838313 -bw.:a66669999---aaaa-aaae Abst html
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers), pp. 4171- 4186. Association for Computational Linguistics, 2019. doi: 10.18653/v 10533//1/n19119//0 URL https://doi. org. 11w53//1//nnnt
1423
Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang. Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, Jing Yi, Weilin Zhao, Xiaozhi Wang, Zhiyuan Liu, Hai-Tao Zheng, Jianfei Chen, Yang Liu, Jie Tang, Juanzi Li, and Maosong Sun. Delta tun- ing: A comprehensive study of parameter efficient methods for pre-trained language models. CoRR, abs/2203 06904, 2022. doi: 10.48550arxx...223..66 069004 URL htttps://deet org/ 10. 0..44855//rr 220.xxi........... 06904
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x 16 words: Transformers for image recognition at scale. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. UtLhttts::/ http::::////peeeeiiieeee net forum? Ed-YichE dNTTy
Emilien Dupont, Arnaud Doucet, and Yee Whye Teh. Augmented neural odes. In Hanna M. Wal- lach, Hugo Larochelle, Alina Beygelzimer, Florence Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 32: Annual Conference Neural Information Processing Systems 2019, NewrIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pp. 34443431444 2019.
Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th Interna- tional Joint Conference on Natural Language Processing, ACL/JCCNP 2021. (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pp. 3816-3830. Association for Computational Lin- guistics, 2021. doi: 10. 10.18633/v122211 1/dl:////22...aclloongggg acl-llnng. 295. URL https: :tapp:://oii. wrw..rrg....... /t.rg//..11533... 18653/ v112021. acl- Long.loon.2000
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In 2016 IEEE Conference Computer Vision and Pattern Recognition, CVPR
10

---

Under review as a conference paper at ICLR 2023
2016, Las Vegas, NV. USA, June 27-30, 2016, pp. 770-778. IEEE Computer Society, 2016. doi: 10.1109/CVP..2066.990 htthstts::// Sattpps:/doi.oogg////.. 111p:////...111/CP.... 2016. 90
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: decoding-enhane bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Viriual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https:
//openzeview.
Werner Horsthemke and René Lefever. Noise-induced transitions in physics, chemistry, and biology. Noise-induced transitions: theory and applications in physics, chemistry, and biology, pp. 164- 200, 1984.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th Interna- fional Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Reseurch, pp. 2790-2799. PMLR, 2019. mllr. wwwppppesssvv... http:///////hhl///e http:hollllyyyy
URL
html
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth Inter- national Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. www//tttounaneee d=nnddeeeeeeeeeeeeeeeeeee
Open 2022. httttttts///:eeeeetet
Patrick Kidger, James Foster, Xuechen Li, and Terry J. Lyons. Neural sdes dimnite-dimensionall gans. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pp. 5453..5553.... 5453-5463. PMLR, 2021. http:: http::://prcceeiine ngs.
mlr. www.ppres//eo v13 hwtps//v19//kke w1t1//19///k11111111 html
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter- efficiet prompt Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 1021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pp. 3045- 3059. Association for Computational Linguistics, 2021. URL https::/
tuning. In
doi: 10.18653/v 1/t/3//1/2221.1 weeplleemlllmaene mainlp-main.243.. 1ttig/00..6653/112000 emn1p ma in.i 243
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Pa- pers). Virtual Event, August 1-6, 2021, pp. 4582-4597. Association for Computational Linguis- tics, 2021. doi: 10.1100.11.. 20118833/v12222 acl- URL 2021. acl.llclloonoo 353,
Xuechen Li, Ting-Kau Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. Scalable gradients and variational inference for stochastic differential equations. In Cheng Zhang, Francisco J. R. Ruiz, Thang D. Bui, Adji Bousso Dieng, and Dawen Liang (eds.), Symposium on Advances in Approximate Bayesian Inference, AABI 2019, Vancouver, BC, Canada, December 8, 2019, volume 118 of Proceedings of Machine Learning Research, pp. 1-28. PMLR, 2019. URL http:// proceedi ngg www.r..pps/.vvvee.m 8/182/1...... html
Xuechen Li, Ting-Kam Leonard Wong, Ricky T. Q. Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In Silvia Chiappa and Roberto Calandra (eds.), The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020. 26-28 August 2020, Online [Palermo, Sicily, Italy], volume 108 of Proceedings of Machine Learning Research, pp. 3870-3882, PMLR, 2020. URL html
ngs m ml wwt.rrrree//v10/111
Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. UMAP: uniform manifold approximation and projection. J. Open Source Softw. 3(29):861, 2018. doi: 10.21 10121000//sss00866 URL http::t/////. :/tttps://do..oo org/ 10. 20112/11555100 joss 00861
11

---

Under review as a conference paper at ICLR 2023
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to- text transformer. J. Mach. Learn. Res., 21:140:1-140:77, 2020. URL
papers/v21/20-0772 htmml
Yulia Rubanova, Tian Qi Chen, and David Duvenaud. Latent ordinary differential equations for iregularly-sampled time series. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelz- imer, Florence d' Alché-Buc, Emily B. Fox, and Roman Garnett (eds.), Advances in Neu- ral Information Processing Systems 32: Annual Conference on Neural Information Pro- cessing Systems 2019, NeurlPS 2019, December 8-14, 2019, Vancouver, BC. Canada, pp. 5321-5331, 2019. URL
neurippp cc/ pttppcappp////////asss...
42
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Im: Training multi-billion parameter language models using model par- allelism. CoRR, abs/1 909.08053, 2019. URL
haapgg/bb/1990 0atp/19990.00333 08053
Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper. Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song. Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. Using deepspeed and megatron train megatron-turing NLG 530b, A large-scale generative language model. CoRR, abs/2201.11990, 2022. URL httpp:: / wawpiv.vvvv..///aa//aaaa/aaaaal. org abs htarrg/bb//20000 11990
K Sobezyk. Stochastic Differential Equations: With Applications to Physics and Engineering, vol- ume 40. Springer Science& & Business Media, 2013.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL
rJ4km2R5t7
Rose E. Wang. Esin Durmus, Noah D. Goodman, and Tatsunori Hashimoto. Language modeling via stochastic processes. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25- 2022. OpenReview.net, 2022. URL/https://openee//i net Sttt/Eorumnn1d=pM0kWW.e
Xiaoqun Wang and Ian H. Sloan. Quasi-monte carlo methods in financial engineering: An equiva- lence principle and dimension reduction. Oper. Res., 59( 95, 2011. doi: 10. 10.1287/opre.1100 1100. 0853. URL PRRL:ttps:///ett /ttttt////ellloogg00...122/////2
1100.
Zonghan Yang and Yang Liu. On robust trefix-tuning for text classification. In The Tenth Inter- national Conference Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL pettts::/oppnzeeee
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. Bitfit: Simple parameter-efficien fine-tuning for transformer-based masked language-moder models.. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.). Proceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 1-9, Association for Computational Linguistics, 2022. doi: (0.18633/1122022aac-hhl
URL
/1tt0//00///////.//////////////0 short.1
Han Zhang, Xi Gao, Jacob Unterman, and Tom Arodz. Approximation capabilities of neural odes and invertible residual networks. In Proceedings of the 37th International Conference on Machine Learning. ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pp. 11086 11095. PMLR, 2020. URL httt::/proceedinng .mlr.
press/vil 9/ whmzg html
Yukun Zhu, Ryan Kiros, Richard S. Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Tor- ralba, and Sanja Fidler. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In 2015 IEFE International Conference on Computer Vision,
12

---

Under review as a conference paper at ICLR 2023
ICCV 2013, Santiago, Chile, December 7-13, 2015, pp. 19-27. IEEE Computer Society, 2015. doi: 10.1109/ICCV22015.111 URLhttps: htt.t:::/d...orr....oooo
2015.11
A BACKGROUND FOR PARAMETER-EFFICHEE TUNING METHODS
The large number of parameters in the PLMs makes fine-tuning impractical, therefore different PETs are proposed to mitigate the problem. The current PETs can be categorized into three groups: addition-based, specification-bar based and reparameterization-bee based (Ding et al. 2022). To verify the generality our method. we include one or two PETs from each category this work, and we give a brief review to these PETs.
Prompt Tuning is an addition-based PET. prepends or appends trainable virtual tokens P Rmxd to each sequence
Raxd to form new imput sequence [P: [x: P] R(++m)xd where n, m are length of original sequence and virtual tokens respectively, the embedding dimension. The virtual tokens P can be either continuous Lester al. (2021) or be restricted to be embeddings of discrete tokens in vocabulary (Gao et al. 2021).
Adapter (Houlsby al., 2019) is an addition-based PET. inserts two-layer MLPs after the atten- tion module and feed-forward module at each layer. Denote h E R4 as the input of Adapter, as the intermediate dimension of Adapter's MLP,, Wa Rexd W Rdxr as the down-pu down-projecto projection and up-projection of Adapter, and as the activation function. Then the computation of Adapter can be formulated as
- W. o a +
BitFit Zaken et al.
2022) is ppecification--asee PET. It specifies the hias terms in layer normal-
ization modules and linear transformation modules as trainable.
LoRA (Hu et al. 2022) is reparameterizzation-aasss PET. It assumes that when training the model, the updates AW for model's pre-trained parameters W Rdxk low-rank, and thus reparam-
eterize the AW of each matrix in attention module with a low-rank decomposition AW B
where B dd A E Br For forward pass Wx, the computation of LoRA can be
written as
h= (W + AW)x Wx + BAx
B PROPERTIES FOR ORNSTEIN-UHLEBBERE BRIDGE
Proposition B.1 (Properties of Omstein-UUllnbber Uhlenbeck Bridge). A Comstenn-Ullebeed X pinned at T 3 0 and x B is the solution to the following SDE:
8 dx, = coth (T t)] X, sinh (T t)] dt + ad X
(13)
where 9 is the diffusion coefficient and a the diffusion for the OU process. The transition proba- bility density function reads as:
sinbla(T (T-)) +1) sinh(g (T -s)
y
p T&9 (t,y 0.0) (s,t
exp
(14)
[0 t)
where
2² sinh(g(T - 1)) sinh(((( s))
(15)
sinh(g(T - 8))
C OTHER RESULTS FOR GLUE EXPERIMENTS
In this section, we present the complete results including OU bridge regularizer for Table and Ta-
ble
We also report the results for Deberta and the results few-shot GLUB for both
13

---

Under review as a conference paper at ICLR 2023
0.70
Bwergg Pafeemmnc
Overggh Pertemvice
0.65
0.54
0.02
0.00
0.54
16
32.
(a) PROMPT TUNING
(b) LORA
FET
PET
Biss
Adapoo
Zrown
3.ron POr
+3rown_SDE
ayorage Pefommnncs
0.64
aacehgg Performance
0.000
0.02
0.64
0.62
0.90
0.58
0.56
32
Shet
Shot
(c) BIAS TUNING
ADAPTER
Figure 3: The average BERT large few-shot GLUE results trained with different PETs under different shots. The results are averaged across 5 different seeds and the error bars indicate the 95% confi- dence. SDE regularizer consistently outperforms the baseline PDF regularizer.
BERT large and Debertastage under 4-, 8-, 16-, and 32-shot. We observe that BERT large cannot give reasonable answers on CoLA and the Matthews correlations are around 0 for all the PETs and all the shots have experienced with. However, the situation gets better for the larger model Debertaxtage- Therefore, we exclude CoLA for BERT large and keep it for Debertastage- We only select the Brown- ian bridge the representative in this section, since the Brownian bridge and Domstein-Uhlenber bridge have no significant difference in Table
In Table 7 and Table 6 report the performance of OU bridge regularizers. The experimental setups are the same as Table [ and Table 2 respectively. The performances between OU bridge we report the performance of Debertagtare on full GLUE datasets. On all four PETs, the SDE regularizer outperforms the PDF regularizer, this consistent with the results we see Table T The results for 4-, 8-, and 32-shot for BERT lunge and Deberta,tage are plotted respectively in Figure and Figure 4 For simplicity, we only plot the average performance for each PET. We report the results for 16-shot experiments for Debertaztarge in Table 9 The setup for the experiment is almost the same the experiment Section 4.3) and the hyper-parameters are listed in Appendix E The SDE regularizer outperforms the PDF regularizer most of the PETs except Prompt tuning. We notice that the SDE regularizer helps Debertägtagee substantially on CoLA for most of the PETs, indicating the SDE regularizer can effectively provide useful guidance when the data is scarce and the task is hard.
and Brownian bridge do have a significant difference. In Table g
D CALCULATION OF CORRELATION SECTION 5
In this section, elaborate how calculate the correlations reported in Table 4 and Table
14

---

Under review as a conference paper at ICLR 2023
0.51
0.90
0.52
0.85
bwerge Portemance
Averggn Parlemunce
0.50
0.59
0.50
0.40
16
32.
(a) PROMPT TUNING
(b) LORA
5,050
FET
0.70
PET
es
Adapoo
Irown
3.ron POr
+brown_00e
Neergg betoommnce
Arcrigg Perfbmmsece
0.60
0.500
0.55
0.525
5.500
0.50
0.45
32
Stet
Shot
(c) BITFIT
(d) ADAPTER
Figure 4: The average Debertäxiarge few-shot GLUE results trained with different PETs under dif- ferent shots. The results averaged across different seeds and the error bars indicate the 95% confidence.
D.1 CORRELATION BETWEEN NUMBER OF SHOTS AND DISTANCE TO BRIDGE
Definition D.1 (Tie). pair of observation ((1,, Bc), (ry, Uy)} defined as tied x; I3 or V/j.
Since we generate the few-shot datasets using 5 random seeds for each shot, each PET has results for each shot. This results in observations with ties, e.g., the two observations for distances on the first seed and second seed for 8-shot {(8, d1). (8, d are tied. To calculate the correlation for data with ties, the Tau-b Kendall's rank correlation is more suitable than Pearson's 's correlation. We therefore report the Kendall's 's rank correlation for the correlation between the number of shots and the hidden states distance to the latent bridges.
D.2 CORRELATION BETWEEN PERFORMANCE AND DISTANCE TO BRIDGE
We mix all the few-shot results for different shots and different seeds to form observations of per- formance and the hidden states distances to bridges, and then calculate the Pearson's correlation.
E HYPER-PARAMETERS
E. TRAINING 9-
use simple 3-layer MLP for 9, in all of our experiments. For PDF regularizer, the output dimen- sions of each layer are 1024, 256, 128, and for SDE regularizer, the output dimensions of each layer 1024, 256, 32. We observe that the running time increases noticeably with the final output dimen- sion for SDE regularizer, we thus choose a smaller one for SDE regularizer. The hyper-parameters for training 9, listed in Table TO
15

---

Under review as a conference paper at ICLR 2023
Table 6: The complete results on GLUE for BERT large under 16-shot setting. We exclude CoLA because all PETs fail to give reasonable result in few-shot setting.
PET
MNLI
QQP
QNLI
SST-2
MRPC
RTE
Average
PROMPT
38. 1..5
53.03.1
51.61.4
70.14.9
50.13.0
48.01.3
51.80.9
+BROWN PDF
38.72.3
54.92.
52.1
75.0
52.82.2
50.83.3
54.0
2.2
1BROWN_SDE
40.60.8
55.42.1
52.91.6
80.0
51.93.6
51.73.1
55.41.5
3.6
AOU_PDF
38.9 38.91.6
54.92.5
51.82.0
71.4g.3
54.32.6
51.53.5
53.81.9
2.0
+OULSDE
40.01.7
56.81.5
53.52.2
82.73.2
52.72.1
52.01.9
56.3u.s
4.5
LORA
48.74.5
59.96.5
53.21.2
90.21.1
53.63.4
64.20.9
61.60.6
+BROWN.PDF
52.01.3
62.72.2
55.13.8
91.3u.1
57.45.0
65.31.5
64.0p.7
2.4
+BROWN
54.10.9
65.51.4
64.35.1
91.20.3
60.23.1
65.81.4
66.80.6
5.2
+OU PDF
51.62.4
62.52.3
55.33.4
91.4n.5
56.95.2
65.6n.4
63.91.1
2.3
+OU SDE
52.92.6
63.1
64.45.1
91.30.3
62.22.7
66.21.1
66.7p.7
5.1
BITFIT
48.41.6
56.06.1
51.72.5
90.8p.8
52.02.3
61.71.4
60.1
+BROWN PDF
48.51.9
56.06.0
53.52.0
91.0u.4
53.92.3
63.21.6
61.0p.8
0.9
+BROWN SDE
52.3u.5
61.22.9
58.81.7
90.8u.4
54.83.0
63.92.5
63.6u.8
3,5
PDF
49.01.8
16.
52.82.2
90.8n.7
53.52.2
62.5
60.8p.s
0.7
+OULSDE
51.81.2
58.42.2
58.94.1
91.1n.4
56.43.6
63.51.4
63.4p.s
3.3
ADAPTER
47.43.7
57.07.2
55.82.9
91.0n.4
55.82.5
62.72.0
61.61.2
+BROWN PDF
49.04.8
58.57.4
56.93.1
91.4n.2
57.24.9
63.23.0
62.71.5
1.1
+BROWN_SDE
52.32.2
62.42.9
64.84.5
91.9n.4
57.34.1
63.81.8
65.41.3
3.8
+OU.PDE 48.44.3
58.47.4
57.73.7
91.30.5
57.13.3
63.01.8
62.61.3
1.0
+OU SDE 48.34.5
62.8
63.84.7
91.10.4
59.82.3
63.61.9
64.90.6
3.3
Table 7: The complete results on GLUE for BERT large- The values the average value of the best performances over three different runs, and the subscripts are the standard deviations. The A column shows the difference of the average performance between the PETs with and without our regularizers.
PET
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
Average
PROMPT
84.4p.1
85.3p.3
91.5p.1
95.5p.1
73.92.4
55.53.4
60.81.s
78.
+BROWN PDF
84.7p.2
85.5
91.8g.6
95.7p.1
75.40.5
56.43.3
61.52.2
78.7
0.6
+OU_PDF
84.70.1
85.4p.1
91.8u.3
95.60.2
76.91.0
57.11.2
60.5s.0
78.9u.2
0.8
+BROWN SDE
EDE
84.9p.2
85.40.1
91.80.4
95.80.3
78.81.2
61.42.9
64.7
80.4p.2
2.3
+OU_SDE
84.7p.2
85.3o.1
92.10.3
95.50.2
80.20.5
61.5a.5
65.92.7
80.7
2.6
LORA
88.8p.1
89.2p.2
93.50.2
95.5
84.60.4
62.8
78.9
84.8p.3
+BROWN PDF
88.9p.1
89.6p.1
93.9n.1
95.60.2
85.
63.7n.5
80.0p.5
85.2n.1
0.4
+OU_PDF
88.9.
89.40.1
93.70.1
95.70.3
86.0p.5
63.6p.6
80.5g.8
85.4p.1
0.6
+BROWN_SDD
88.9p.1
89.5p.1
93.70.1
95.70.1
86.51.2
63.90.4
80.9..s
85.60.1
0.8
+OU -DE
88.8o.o
89.5o.1
93.70.2
95.70.3
86.31.2
63.7p.6
80.
85.40.1
0.6
BITFIT
87.90.2
87.6p.1
92.70.2
95.60.1
79.42.3
60.2p.8
77.0
82.90.3
+BROWN_PDF
87.90.1
87.8p.o
93.00.2
95.70.1
83.1o.8
60.3p.6
78.30.9
83.70.2
0.8
+OULPDF
87.90.1
87.8p.1
93.0u.1
95.7p.1
82.60.8
59.81u
78.81.4
83.6u.4
0.7
FRROWN_SD
87.9u
87.76.0
92.8u.1
95.70.1
83.30.8
61.112
77.71.5
83.8u.3
0.9
+OU SDE
87.9
87.6p.1
92.7p.1
95.8
83.7p.s
60.8
77.10.9
83.7
0.8
ADAPTER
88.8p.1
89.6p.3
93.70.1
95.6p.1
83.60.1
60.41.
79.51.2
84.5u.3
+BROWN PDF
89.0o.1
89.70.2
93.80.2
95.80.1
86.51.1
62.60.7
83.20.2
85.80.2
1.3
+OU PDF
88.9p.1
89.70.0
93.8g.1
95.80.1
86.80.6
61.9n2
82.00.6
85.6n.1
1.1
+BROWN SDE
88
88.9p.1
89.8p.1
93.9
95.8p.1
85.90.4
62.31.8
82.20.2
85.5p.2
1.0
+OU_SDE
88.9p.1
89.8p.1
93.7
95.70.1
85.90.7
62.5
82.70.5
85.60
1.1
16

---

Under review as a conference paper at ICLR 2023
Table 8: The results on GLUE for Debertaclarge.
PET
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
Average
A
PROMPT
87.2p.1
86.50.1
9
96.8p.1
75.42.5
64.2
78.83.5
83.2
+BROWN_PA PDF
87.60.1
86.8p.3
94.2g.1
96.80.1
80.32.9
65.5
79.80.5
84.5p.6
1.3
+BROWN_SDE
87.60.1
86.8p.1
94.0p.1
96.8p.1
84.40.6
64.8p.5
79.51.3
84.80.2
1.6
LORA
91.1p.1
90.3p.1
15.1a.1
96.8p.1
88.70.7
68.01.3
83.41.1
87.6p.3
+BROWN PDF 91.1u.0
90.5
95.2g.0
97.0p.1
90.10.8
68.6p.8
85.91.5
88.3u.2
0.7
+BROWN
91.1p.1
90.40.0
95.1o.0
96.90.2
90.50.6
69.61.1
85.60.9
88.5p.
0.9
BITFIT
90.0u.1
88.40.0
95.00.0
96.60.1
87.30.6
66.9u.2
82.40.6
86.7u.1
+BROWN PDF
90.20.0 a
8..3
95.0g.1
96.60.1
89.8p.5
67.9
82.90.6
87.2
0.5
+BROWN SDE
10.1
88.3p.0
94.8g.0
96.60.1
90.40.5
67.9p.4
83.80.5
87.4p.1
0.7
ADAPTER
91.10.1
90.0p.1
95.20.0
96.80.2
87.90.5
68.81.8
85.00.6
87.80.2
+ +BROWN PDF
91.2p.1
90.0p.0
95.30.1
96.90.2
89.2g.8
70.11.0
86.91.5
88.5p.5
0.7
BBROWNSSDE
91.2p.2
90.1
95.20.0
96.90.2
90.30.9
70.81.1
86.31.4
88.7
0.9
Table 9: The results on GLUE for Debertastage under 16-shot setting.
PET
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
Average
PROMPT
34.41.2
53.25.1
51.71.7
73.38.7
50.23.2
2.5
52.2
45.42.1
+BROWN.PDD
35.80.9
57.72.4
53.51.s
87.53.0
52.21.9
2.82.9
54.11.8
49.1o.s
3.7
+BROWN SDE
35.91.6
59.66.4
53.11.9
82.14.2
55.41.4
2.93.0
54.11.3
49.01.1
3.6
LORA
43.13.6
68.42.9
60.15.3
91.81.1
57.61.5
3.13.8
56.62.3
54.41.0
+BROWN PDF
52.13.1 70.22.9
73.36.3
91.71.1
59.5
14.25.6
60.41.2
60.21.2
5.8
+BROWN SDE
49.64.3
70.61.5
72.46.0
90.71.2
59.84.5
28.91.5
60.6g.1
61.8p.5
7.4
BITFIT
41.93.8
67.72.6
60.31.2
91.80.8
54.92.5
9.42.4
57.62.0
54.80.8
+BROWN PDF
45.23.7
70.31.2
65.46.7
90.9g.s
55.62.1
8.22.5
59.62.3
56.5u.T
1.7
+BROWN-SDE
45.7
69.22.3
69.26.3
89.71.3
57.84.
24.24.6
59.42.8
59.31.2
4.5
ADAPTER
43.12.9
67.72.7
55.95.3
91.10.9
56.12.1
8.65.6
59.02.3
54.51.4
+ +BROWN_PDE
50.73.0
70.11.6
70.9s.2
90.61.7
57.64.3
16.17.4
60.6g.9
59.50.8
5.0
+BROWN BDE
47.11.6
72.01.1
71.3
91.01.0
59.74.5
26.45.5
60.0s.7
61.1g.6
6.6
Table 10: Hyper-parameters for training By
PDF
SDE
Learning rate
le-3
le-3
Weight decay
0
Batch size
128
128
Grad norm
1.0
1.0
Max steps
100k
500k
Warmup ratio
0.01
0.01
E.2 TRAINING PETS ON FULL-SET GLUE
We run all the experiments for 50k steps, and evaluate on the development set every 1 steps. For BERT large, we use 32 as the batch size while for Debertastane- we use 16 the batch size. We choose learning rate le-3 for Prompt tuning for both PLMs, and le-4 for other PET' for both PLMs. We use 0.01 weight decay, 1.0 maximum gradient norm and no learning rate warm-up for all the experiments. We search the best regularization strength a in (0.01, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] for PDF regularizer, and in f0.0001, 0.0005, 0.001, 0.005, 0.01, 0.05, 0.1. 0.2, 0.5, 1.0) for SDE regularizer. The best a for PDF regularizer are listed in Table [] and best a for SDE regularizer listed in Table
17

---

Under review as a conference paper at ICLR 2023
Table 11: Best for PDF regularizer on full-set GLUE
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
BERT large
PROMPT
0.05
0.3
0.01
0.2
0.8
0.4
0.4
LORA
0.2
0.8
0.5
0.1
0.1
0.5
0.3
BIAS
0.05
0.3
0.6
0.05
0.8
0.05
0.7
ADAPTER
0.8
0.01
0.7
0.1
0.6
0.4
0.3
Deberiasiage
PROMPT
0.05
0.1
0.2
0.8
0.7
0.3
0.7
LORA
0.1
0.1
0.05
0.01
0.1
0.01
0.2
BIAS
0.05
0.01
0.01
0.01
0.8
0.01
0.8
ADAPTER
0.3
0.01
0.01
0.01
0.01
0.6
0.1
Table 12: Best for SDE regularizer full-set GLUE
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
BERT large
PROMPT
0.005
0.005
0.0005
0.0001
1.0
0.2
0.05
LORA
0.0005
0.01
0.01
0.01
0.005
0.0005
0.0005
BIAS
0.001
0.001
0.005
0.0001
1.0
1.0
0,005
ADAPTER
0.001
0.001
0.0005
0,001
0.005
0.5
0.005
Debertaztorge
PROMPT
0.0001
0.001
0.0001
0.0001
0.2
0.0001
0.0001
LORA
0.0005
0.05
0.0001
0.0001
0.1
0.005
0.0001
BIAS
0.0001
0.0005
0.0001
0.0001
0.05
0.2
0.001
ADAPTER
0.001
0.0005
0.0001
0.0005
0.1
0.001
0.0005
E.3 TRAINING PETS ON FEW-SHOT GLUE
We run all the experiments for Ik steps, and evaluate on the development every 50 steps. For all the shots for both regularizers and both models, we use batch size of 2. Other hyper-parameters are kept the same as the experiments on full-set GLUE.
F PERFORMANCE OF REGULARIZERS TRAINED WITH TINY CORPUS
Although the pre-training corpus train our mapping I-y the training is actually fast and data-efficient. We show that when using only 10,000 documents in the pre- pre-training corpus (about 0.1% of the corpus), the obtained regularizers still perform great and are comparable the regular- izers trained on the whole pre-training corpus. We train the mapping 9, for 5,000 iterations with 128 batch size. On single NVIDIA A100 GPU, the training can be done hour for PDF regularizer, and hours for SDE regularizer. The cost of training our regularizer is quite small compared to the resources required for pre-training.
We conduct the same experiments as Section 4.2 and Section 4.3 with the regularizers trained with tiny corpus. The results are presented in Table and Table respectivell.
On full-set GLUE, the PDF regularizer performs even better three out of four PETs, and although its performance affected on Adapter, it still outperforms the vanilla Adapter. The SDE regularizer slightly affected three out of four PETs, but it still brings substantial improvements all the PETs.
On few-shot GLUE, the impact of the shrinkage of the corpus is relatively obvious. But overall, the regularizers still performs great on all the PETs. The drop in performances are relatively small compared to the boost they bring to vanilla PETs.
18

---

Under review as a conference paper at ICLR 2023
The results on GLUE for BERT large with regularizers trained on 0.1 1% of the pre-training corpus. A whole is the difference between the the average performance in this table and the average
Table 13:
performance Table
PET
MNLI
QQP
QNLI
SST-2
MRPC
CoLA
RTE
Average
A
Atole
PROMPT
84.40.1
85.3p.a
91.5p.1
95.5..1
73.9
55.5
60.8
78.
+BROWN PDF
84.7a.0
85.4..1
92.1o.1
95.80.2
75.81.4
56.6p.9
61.1s.0
78.8
0.7
+0.1
+BROWNSDE
84,8a.2
85.4
92.1..1
95.80.2
79.01.3
59.8g.2
65.5
80.31.2
2,2
-0.1
LORA
8
89.2g.a
93.5p.2
95.5p.1
84.6y.4
62.8,.6
78.9 78.9.
84.80.3
+BROWN.
88.9
89.5p.1
93.8p.1
95.80.2
86.0g.s
64.40.6
80.4p.3
85.5n.1
0.7
+BROWN_SDE
88.9
89.4p.1
93.80.1
95.60.2
86.8
63.5u.9
80.9p.s
85.50.2
0.7
-0.1
BITFIT
87.9..2
87.6p.
92.7p.a
95.60.1
79.42.3
60.2g
77.0
82.90.3
+BROWN.PDD
87.9..1
87.7p.1
92.8p.1
95.70.2
83.6a.7
61.2n.2
78.71.1
84.0n.1
1.1
+0.3
+BROWN_SDE
87.9..1
87.7p.1
92.8p.1
95.76.1
82.8u.7
61.1o.8
77.4p.6
83.60.3
0.7
-0.2
ADAPTER
88.8a.1
89.6n.a
93.70.1
95.60.1
83.60.1
60.41.
79.51
84.5n.3
1BROWN_PDE
88.9..1
89.8p.1
93.80.2
95.90.2
85,4g.s
60.9p.s
82.71.s
85.3p.2
0.8
-0.5
+BROWN
88.9
89.8p.1
93.70.1
95.80.3
85.7g.5
61.9u.9
82.4p.a
85.5p.2
1.0
0.0
Table 14:
The results GLUE for BERT large under 16-shot setting with regularizers trained on 1% of the pre-training corpus. Ahhole is the difference between the the average performance in this table and the average performance in Table
PET
MNLI
QQP
QNLI
SST-2
MRPC
RTE
Average
A A
PROMPT
38.11.5
53.0s.1
51.61.4
70.
50.13.0
48.01.3
51.80.9
+BROWN.PDF
38.41.5
54.62.5
52.41.4
73.16.0
54.01.4
51.11.7
53.91.1
2.1
-0.1
+BROWN_SDE
39.31.1
56.23.0
52.81.1
80.87.1
53.06.0
51.02.5
55.50.9
3.7
+0.1
LORA
48.74.5
59.9s.a
53.21.2
90.21.1
53.63.4
64.2p.a
61.6n.6
+BROWN
51.42.2
62.01.a
54.83.1
91.20.3
57.44.0
65.7p.7
63.7u.9
2.1
-0.3
+BROWN.SDE
53.21.9
65.41.7
64.2..u
91.2u.3
61.4g.8
66.3u.7
66.9u.7
5.3
-0.1
BITFIT
48.41.6
$6.06.1
$1.72.5
90.80.6
52.0g.3
61.71.4
60.1
+BROWN PDF
49.11.8
56.06.a
53.12.4
91.1u.2
54.02.0
62.81.a
61.0u.7
0,9
0.0
+BROWN
51.41.4
60.11.7
56.71.8
91.1u.3
57.34.5
63.01.2
63.2u.7
3.1
-0.4
ADAPTER
47.4s.7
$7.07.2
55.82.9
91.0u.4
55.82.5
62.72.0
61.61.2
+BROWN PDF
48.14.0
58.36.9
57.95.6
91.40.4
57.73.6
63.0s.o
62.71.0
1.1
0.0
+BROWN.SDE
50.32.2
61.0s.1
63.44.4
91.50.3
58.42.2
64.31.9
64.81.1
3.2
-0.6
G SPEED OF THE REGULARIZERS
As we have mentioned in Section the PDF regularizer only incurs negligible computational cost. In this section, we present the plot of step-metric curve on full-set GLUE in Figures 5 6 7 and g
On different PETs, the regularized PETs with PDF regularizer has similar running time the vanilla PETs. On the two large datasets, QQP and MNLI, regularized PETs with SDE regularizer take about 2 to 3 times longer to achieve the best performance than vanilla PETs. However, on medium-sized (QNLI, SST-2) and small datasets (CoLA, MRPC, RTE), the time to achieve the best results with SDE regularizer is comparable to vanilla PETs.
Overall, the PDF regularizer can effectively improve the performance of PETs without introducing much computational cost. In scenarios where there is relatively more focus on the inference perfor- mance of PETs and less concern about the slightly longer training time, or when the dataset is small, SDE regularizer should be a good choice.
19

---

Under review as a conference paper at ICLR 2023
0.43
0.342
6775
6.75
6:00
0413
oran Each www.eeeneeee
2005 12:00 LSVER 20000 20000
2000
(a) QQP
(b) MNLI
(c) QNLI
(d) SST-2
Modd
80
T0
Twres
Team
(c) CoLA
(f) MRPC
g RTE
Figure 5: Time-Metrie curve for regularizers prompt tuning.
0.97
0.0
6315
372
QQP
(b) MNLI
(c) QNLI
(d) SST-2
Brown
OUE
0.44
1009 1909 2008 .000
5000 19000
2010
AP000 2300
(e) CoLA
(I) MRPC
(g) RTE
Figure 6: Metei curve for regularizers on LoRA.
20

---

Under review as a conference paper at ICLR 2023
1.8
Mond
Mate
FOr
FOL
935
6330
1380
2000
50.00
7800
1980
7940
QQP
(b) MNLI
(c) QNLI
(d) SST-2
0.82
032
2008
2008
Frems
Trams
(e) CoLA
(I) MRPC
gg
RTE
Figure 7: Time-Metric curve for regularizers on BirFit
0.57
Madel
ROde
Paack
3.52
2004
J000 15000
J008
2000
5850
10000
2008 2000
6050
2010 20000
5050
20805 20000
(a) QQP
(b) MNLI
(c) ONLI
(d) SST-2
POOI
0.LL
SUL SROW
6.14
6008
2003 5000 2000 13000 2008 2000
J0000
12000
2000
2000
1200
20000
(e) CoLA
(f) MRPC
gg) RTE
Figure 8: Time-Metric curve for regularizers on Adapter.
21


Reviews:
[{"review_id": "ymH78zDeLmI", "reviewer": "Reviewer_PqBu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Since recent pre-trained language models are large, it is common to fine-tune small amount of parameters instead of tuning all the parameters when we would like to adapt the pre-trained model for downstream tasks (e.g. text classification etc.). Many methods have been proposed such as Prompt tuning, Adapter, BitFit and LoRa (summarized in appendix A). For example, Adapter, one of the simplest approach inserts extra layers (called adapter layers) in transformer blocks and train those layers with supervision from downstream tasks.\n\nThe straight forward approach is to directly optimize the downstream task loss, but instead authors proposed a regularization method. The proposed regularization objectives encourage hidden states of each layer to be able to approximate low-rank vector representation (PCA) of target tokens following the diffusion process explained in sec. 2.2 (and parameterized as in sec. 3). The proposed objective is added to the task objectives with some weights.\n\nThey evaluated their regularization method on fine-tuning BERT_large model with four parameter tuning approaches (Prompt tuning, Adapter, BitFit and LoRa) and show that the regularization lead to performance improvements on GLEU benchmarks in both full-set and few-shot scenario.", "summary_of_the_review": "Although the proposed method show improvements, it requires extra data to pre-train the parameters than baselines. And it is hard to make fair comparison with existing methods especially for the few-shot setup. Also, the use of stochastic bridges is not justified conceptually and empirically. For these reasons, I believe this paper is not ready for publication.", "strength_and_weaknesses": "Strength\n- Since the proposed regularization is not dependent of parameter tuning methods, it can be used for many parameter tuning methods.\n- The experiments shows improvements on GLEU benchmarks in both full-set and few-shot scenario.\n\nWeakness\n- It was not clear what optimal control perspective implies. It's common to use regularization / prior / pre-training in machine learning but the new perspective implies something special (which lead to the proposed method)?\n- The method assumes that the pre-training corpus is available, but previous works do not assume so.\n- The method looked like pre-training of newly introduced parameters (effective for few-shot setup). Is there any justifications why the stochastic bridges is necessary?\n  - What will be the difference between stochastic bridges and pre-training the new parameters on PLM objective?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "PI3kzogS8CX", "reviewer": "Reviewer_69ms", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a class of regularization objectives that are used to improve the effectiveness of parameter-efficient tuning methods (PETs) for pre-trained language models. Specifically, the objective consists of two main components: (i) a (potentially learnable) diffusion bridge defining a target diffusion process; (ii) a mapping function from every layer's output of the language model to a trajectory that approximates the diffusion bridge. This paper uses a fixed diffusion bridge defined by applying PCA to the output word embeddings. Experiment results show that the proposed regularizer is able to improve the performance of existing PETs.", "summary_of_the_review": "In summery, I tend to vote for acceptance since the proposed regularizer makes clever use of target diffusion processes to improve the fine-tuning performance. However, the proposed method requires pre-training corpus, which could add significant computation cost. It is also not well discussed how much pre-training corpus is needed to receive significant performance gain.", "strength_and_weaknesses": "Strength:\n\n- The diffusion bridge forces the latent representation of each layer to contain more information of the target, and can be viewed as a shortcut for learning better latent representations tailored to different tasks.\n\n- The authors demonstrate that task performance is correlated with the distance to the diffusion bridge, which serves as an empirical evidence of the effectiveness of the proposed regularizer.\n\nWeaknesses and questions:\n\n- One weakness of the proposed method is that pre-training corpus is required to train the mapping $g_{\\gamma}$. The problem here is two-fold: (i) pre-training corpus could be hard to access and (ii) since it is much larger than datasets of downstream tasks, training $g_{\\gamma}$ could be computational demanding. For example, I would like to know whether the proposed method is still effective if we only have access to a small subset of the pre-training corpus.\n\n- Following the previous comment, the computation cost of training $g_{\\gamma}$ should be discussed. Furthermore, it would be nice if the authors can provide additional experiments by constraining the GPU time of the baselines and the proposed approach to be the same.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "HkXDqK-MhrM", "reviewer": "Reviewer_CmgL", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "\n\nThis paper proposes a novel framework to do parameter efficient fine-tuning method for PLM (pre-trained language models). The basic idea is to hypothesize that there exists a map from the hidden states in each layer to a lower dimensional latent space, where the trajectory of this lower dimensional map from the zeroth layer to the final layer is from a diffusion bridge that ends at the PCA reduced dimension points of each output target word in the final layer.  The authors propose an interpretation of this as the running cost to augment the usual final terminal cost of the loss function. \n", "summary_of_the_review": "Overall, this paper proposes an interesting idea for a regularizer for fine-tuning a pre-trained language model (PLM). The most interesting part about this is to use the frozen model to constrain the dynamics of the hidden states to follow a distribution similar to those in the pre-training corpus (via a dimensionality reduction). The authors also allude to optimal control connections for this idea, which is not particularly significant in my view. However, the clarity of presentation is still somewhat unclear as I write above in more detail. \n\nThe experimental results seem promising, but the claims about a novel discovery related to the hidden states forming a latent bridge might not be fully convincing. Overall, the paper has some merits in terms of novelty and motivation, but the technical details and the significance of the empirical results are not totally convincing.\n\n", "strength_and_weaknesses": "Strengths:\n\nThe key idea is to learn a frozen non-linear dimensionality reduction map that recovers the dynamics of a stochastic bridge on the pre-training corpus and then using it in the fine-tuning stage to define an additional regularization cost. Overall, the proposal of using a diffusion bridge with an endpoint derived from the word embedding at the final layer in PLM (a projected version of it) seems novel / interesting.  Justifying the novel fine-tuning objective as a regularization cost seems intuitively plausible, even if the other motivations like considering it as running cost versus terminal cost appear to be a superficial connection. The authors also conduct experiments to show that this scheme can help marginally improve the final results in practice. In addition to evaluating the benchmarks, the authors also conduct some analysis of the hidden states of the various methods in terms of the clustering properties of a reduced dimension visualization.\n\nWeaknesses:\n\n- Clarity of the writing is quite unclear and leaves a lot of details to imagination.  Please see the next section for concrete suggestions.\n-  The authors claim that they \"reemphasize\" a \"running cost\" in addition to the terminal cost (which is simply the model loss), however it is unclear why this running cost is important for the overall objective, especially given that this is a cost that the authors define, rather than inspired or proposed from a pre-existing consideration or loss.  \n- It also seems somewhat arbitrary that this new regularization cost is specific to parameter efficient tuning, but it does not seem specific to this setting (e.g. what aspect of going from general finetuning to PET is specific to the method proposed here?)\n- Optimal control formulation is often mentioned for the motivation, but the connection is limited to that and not really used in the technical contents of the paper.\n- The significance of hidden state in vanilla PETS \"approaching the latent bridges\" is unclear to me. Specifically, given that the final reduced dimension representations of the embeddings are estimated from the pre-trained corpus, wouldn't a better fit to the end points automatically imply a better prediction accuracy? \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject