PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent
Authors: Guangliang Liu
Zhiyu Xue
Xitong Zhang
Kristen Johnson
Rongrong Wang
Venue: EMNLP 2023 Main (2023)
Keywords: language model
fine-tuning
pac-bayesian bound
perturbed gradient descent

Abstract: Fine-tuning pretrained language models (PLMs) for downstream tasks is a large-scale optimization problem, in which the choice of the training algorithm critically determines how well the trained model can generalize to unseen test data, especially in the context of few-shot learning. To achieve good generalization performance and avoid overfitting, techniques such as data augmentation and pruning are often applied. However, adding these regularizations necessitates heavy tuning of the hyperparameters of optimization algorithms, such as the popular Adam optimizer. In this paper, we propose a two-stage fine-tuning method, PAC-tuning, to address this optimization challenge. First, based on PAC-Bayes training, PAC-tuning directly minimizes the PAC-Bayes generalization bound to learn proper parameter distribution. Second, PAC-tuning modifies the gradient by injecting noise with the variance learned in the first stage into the model parameters during training, resulting in a variant of perturbed gradient descent (PGD). In the past, the few-shot scenario posed difficulties for PAC-Bayes training because the PAC-Bayes bound, when applied to large models with limited training data, might not be stringent. Our experimental results across 5 GLUE benchmark tasks demonstrate that PAC-tuning successfully handles the challenges of fine-tuning tasks and outperforms strong baseline methods by a visible margin, further confirming the potential to apply PAC training for any other settings where the Adam optimizer is currently used for training.

PAC-tuning: Fine-tuning Pretrained Language Models with PAC-driven Perturbed Gradient Descent
Guangliang Liu", Zhiyu Xuet, Xitong Zhang", Kristen Maric Johnson" and Rongrong Wang"
Michigan State University
UC Santa Barbara
(liuguan5, zhangxit t,, kritktritt wangron6 (enngrono@msuu... zhiyuxue@ucsb. edu
Abstract
NLP, achieving state-of-the-art results in various tasks. There are two main fine-tuning approaches: parameter- tuning and prompt-tuning. Parameter- tuning considers a PLM as a feature extractor and tries to update the complete PLM with a small learn- ing step (Jiang et al., 2020; Gunel et al., 2020). By contrast, prompt-tuning aligns downstream tasks with the objective of language modeling through inserting prompts with or without task demonstra- tions into the original sample and asking the PLM to predict the next token according to the prompted input context (Gao et al., 2021; Gu et al., 2022: Hu et al., 2022b). In the context of few-shot learning, parameter-tun tenierr-tttnngggn over neural model with up to billions of parameters is non-trivial task (Zhang et al., 2020; Dodge et al., 2020). A significant chal- lenge is that the training process unstable (Mos- bach al., 2020; Lee et al., 2019): given only few samples from downstream tasks, the over- parametcrization nature of a PLM Icads to issucs such as overfitting and forgetting (He et al., 2021; Kirkpatrick et al., 2017). Existing methods mainly address these challenges by applying data augmen- tation (Zhou al., 2022; Wu et al., 2022; Kumar et al., 2019), regularization (Zhu al., 2019; Yu et al., 2021; Aghajanyan et al., 2020; Jiang et al., 2020) and network pruning (Xu al., 2021).
Fine-tuning pretrained language models (PLMs) for downstream tasks is large-scale optimization problem, which the choice of the training algorithm critically determines how well the trained model can generalize unseen data, especially in the context of few-shot learning. To achieve good gener- alization performance and avoid overfitting, techniques such data augmentation and pruning are often applied. However, adding these regularizations necessitates heavy tuning of the hyperparameters of optimization algorithms, such as the popular Adam opti- mizer. this paper. we propose two-stage address fine-tuning method, PAC-tuning, this optimization challenge. First, based on PAC-Bayes training, PAC-tuning directly minimizes the PAC-Bayes generalization bound to learn proper parameter distribution. Second, PAC-tuning modifies the gradient by injecting noise with the variance learned in the first stage into model parameters during training, resulting in variant of perturbed gradient descent (PGD). In the past, the few- shot scenario posed difficulties for PAC-Bayes training because the PAC-Bayes bound, when applied to large models with limited training data, might not be stringent. Our experimental results across 5 GLUE benchmark tasks PAC-tuning' successfully handles the challenges of fine-tuning tasks and outperforms strong baseline methods by visible margin, further confirming the potential apply PAC training for any other settings where the Adam optimizer is currently used for training.
demonstrate
From the perspective of machine learning the- ory, data augmentation, regularization, and prun- ing are all used during training as generalization enhancers. Other well-known generalization en- hancers include weight-decay and dropout. Per- haps a little surprisingly, different choices of learn- ing rates (Li et al., 2019) and mini-batch sizes (He et al., 2019) also affect generalization. A less well- known enhancer to the NLP community is noise in- jection, implemented by the PGD (Perturbed Gradi- Descent) algorithm (Orvieto al., 2022, 2023). Theoretically, PGD shown to effectively help the algorithm escape spurious local minima (Zhou et al., 2019) and saddle points (Jin et al., 2021),
1 Introduction
Since the emergence of pretrained language mod- els (PLMs), e.g., BERT (Devlin et al., 2018 8) and GPT-3 (Brown et al., 2020), fine-tuning of such pre- trained models has been the de-facto pipeline for
!Our implementation is publicly available at http://ghhaa..cm//SSLAAPPcSSSSCC

---

due to an implicit regularization on the trace of the Hessian matrix.
layer-wise learning rate and instability of cross- entropy loss, in order to recommend techniques for improving the final generalization performance. contrast to fine-tuning methods which require an update for the model parameters, another research line explores prompting based methods; Prefix- tuning (Li and Liang, 2021) is representative one. A straightforward solution for few-shot tasks is to generate more data via data augmentation (Arthaud et al., 2021; Feng et al., 2021). To grapple with the forgetting issue of fine-tuning PLMs, trust-region- based methods define a trustworthy region con- straining the change of parameters in each update step. Based on the lottery-ticket hypothesis be- hind PLMs (Frankle and Carbin, 2018), parameter- tuning methods that only update the sub-network of PLM have also been proposed (Ben Zaken et al., 2022). All these methods require heavy hyperpa- rameter searches and minimization of the training error, instead of directly optimizing over the gener- alization error.
In this paper, instead of searching for the op- timal combination of a basket of generalization enhancers, we follow an alternative training frame- work, PAC-Bayes training (Rivasplata et al., 2019), which provides a more straightforward way of im- proving the generalization the network is trained towards directly minimizing the generalization er- ror characterized by the PAC-Bayes bound (Maurer, 2004) instead of minimizing only the training loss. Although PAC-Bayes bounds are classical bounds in learning theory, leveraging them for training is relatively new. This is likely due to concerns that the PAC-Bayes bounds suffer from the curse of dimensionality (Dziugaite and Roy, 2017a; Foong et al., 2021); therefore they are unlikely to be effec- tive on modern neural networks that are large and deep. However, recent studies (Rivasplata et al., 2019; Zhang et al., 2023) have shown that this view might be too pessimistic, and that the PAC-Ba BAC-Bayes bound could be rather effective in training modern convolutional neural networks.
PAC-Bayes Training means training machine learning models by minimizing the PAC-Bayes upper bound. In contrast to empirical risk min- imization, PAC-Bayes training is more straight- forward in terms of improving generalization by minimizing the upper bound of generalization ror. McAllester (1998) trains stochastic neural network on an MNIST dataset by minimizing a non- vacuous PAC-Bayes bound. The PAC-Bayes train- ing with BackProp proposed by Rivasplata et al. (2019) trains shallow probabilistic neural networks and certifies their risk by PAC-Bayes bound on the MNIST dataset. Zhang et al. (2023) introduced Auto-tune PAC to train various neural networks, such as ResNet and GNN, through optimizing both the prior distribution variance and posterior dis- tribution variance of parameters. Auto-tune PAC leverages larger model and larger dataset, includ- ing ResNet34, 1enseNet121, and the CIFAR 100 dataset, and the authors test a GNN on a smaller dataset with only 20 nodes per class. Previous works overlook confidence difference between pre- trained layers and adaptation layers, this is the main reason that those works not be applied to PLMs. We, however, take the confidence difference into account by learning the noise level associated with pretrained layers and adaptation layers separately.
This paper explores the potential of PAC-Bayes training on even larger models, namely, the PLM. We consider the most challenging task in the per- spective of PAC-Bayes training, the fine-tuning task, as it amounts to using an extremely small training dataset to tune a PLM with millions of parameters. For this setting, we propose a novel, efficient implementation of PAC-Bayes training, called PAC-tuning, which consists of two stages. The first stage learns the noise variance and updates the PLM's parameters by minimizing a PAC-Bayes upper bound. The second stage implements noise injection training with the noise variance learned from the previous stage. We validate the effec- tiveness of PAC-tuning with few-shot text clas- sification tasks extracted from the GLUE bench- mark. The overall good performance of PAC- tuning suggests promising potential for leveraging PAC-Bayes training for fine-tuning much larger PLMs, even during the pretraining process. To the best of our knowledge, PAC-tuning is the first work of its kind in terms of improving PLM fine-tuning by PAC-Bayes training.
2 Related Works
Few-shot learning with PLMs has been compre- hensively studied by Zhang et al. (2020) to under- stand the influence of various factors, such as the
Perturbed Gradient Descent (PGD) implic- itly regularizes the trace of the Hessian matrix to push the model towards a region of the loss land-

---

scape with larger flatness, which claimed to be measurement of generalization (Foret et al., 2020; Jiang et al., 2019). Zhou et al. (2019) proves that PGD can help a two-layer convolutional neural network model escape a spurious local minimum and converge to global minimum. A similar generalizatiion-nnancee benefit of PGD is also validated by Jin et al. (2021): given PGD, neu- ral network models converge to second-order stationary points and avoid saddle points. While ex- isting PGD works assign isotropic noise models, causing training loss explosion, PAC-tuning avoids this problem by injecting parameter-wai wise noises to PLMs.
hold), and for any neural network type. They char- acterize the generalization error of a trained model. Here, the weight of the neural network, m is the number of training samples, 2 and P are arbi- trary pairs of prior and posterior distributions, KL is the Kullback- divergence measuring the distance between two distributions, and D is the training data distribution. When the PAC-Bayes bound is nonvacuous, minimizing the bound effec- tively reduces the generalization error. In several recent works (Rivasplata et al., 2019; Zhang et al., 2023), optimization algorithmss have been proposed to find the minimizer of J 2,,00 ()) when and @ are taken to be multivariate Gaussian distribu- tions. This provides an automatic way to learn the optimal noise levels (which are the variance of 2) that reflect the different confidence levels of each parameter in the model 0.
3 Method
This section presents our proposed method, PAC- tuning, an implementation of PAC-BByes training for parameter-based fine-tuning of PLMs. Sec- tion 3.2 introduces PAC-Bayes training and the PAC-Bayes bound, followed by Section 3.3 which describes perturbed gradient descent. The moti- vation to assist PGD with PAC-Bayes training is presented in Section 3.4, and we explain the details of PAC-tuning in Section 3.5.
3.3 Noise Injection and Perturbed Gradient Descent (PGD)
The KL term in the LPAC may suffer from two pos- sible issues: (1) it could be difficult to compute and (2) it could be too large to allow the training loss to approach O. As result, it is a common practice to ignore the LPAC term in the PAC-Bayes bound and simply minimize Lirain In the simplest case, we use isotropic Gaussian noise, N (0, n/J), with mean and noise level r) as the posterior distribution, and then Luaan reduces to:
3.1 Problem Setup and Notations
Let 0 be the parameters of the PLM. We replace the head layer of the PLM with a onc-layer fully- connected neural network parameterized by w. De- noting the PLM classifier as f. we consider 0 and w as vectors for simplicity. Let f(:; (.:,,ww) w) be the loss function, e.g., the cross-entropy loss. An indi- vidual sample is represented with (z, y) where is the input data and is the associated label.
1 m
m E Mo. mT) (1)
Luviin
This can be interpreted as the original training loss with noise injected into the model parameters, and our goal is to minimize its expectation.
3.2 PAC-Bayes Training and the PAC-Bayes Bound
The algorithm that minimizes Lurain is called Per- turbed Gradient Descent (PGD), which injects ran- dom noise into the model before computing the gradient and removes the added noise after the gra- dient update.
The idea of PAC-Bayes training arises from mini- mizing the PAC-Bayes bound JO, 2, Lurinn
Teac of the following type:
generalization error
Algorithm 1 describes the application of Per- turbed Gradient Descent to the PLM. Specifically, in line 2, noises and 72 are sampled from stan- dard Gaussian distribution whose dimension is the same 0 and w (we refer readers to the confidence difference issue in Section 3.5), respectively. Next, rescale the sampled noises by 7/ and 7)2 and inject them into the parameters of the PLM f produce noisy parameters 0' and as shown line 3. Parameters are then updated according to
Eg (x.
1
m
log 1 (L(
Egg (1r,, Mii 0)
171
2m
Lowc
Aaain
BAC-Bayes bounds are prohabilistic bounds that hold with high probabilities, i.c., 1 - (where is the probability that the upper bound does not

---

Algorithm 1: Perturbed Gradient Descent
tuning, train the model a second time, and there- fore we expect the pretrained part be updated less in the second round. In other words, 0 should not change much since the PLM is assumed to be accu- rate enough and we generally use small learning rate to update 0 (Zhang et al., 2021). In contrast, the learning rate for should be much larger be- cause we are less confident about it. We name this as the confidence difference issue. Recall in Sec- tion 3.4, we explained how the noise level reflects our confidence in the target parameters. Therefore, wc are motivated to use different noise levels as well as learning rates for and w. In turn, the KL term in the LPAC would consist of two parts:
Sample (a: v) from training dataset
Sample noise from a Gaussian distribution
72 N (O,T))
Rescale and inject noise
8',w' A + / TT,
n, + T12
Update parameters f,w
@L(z,d w²)
aM w/)
0 - 0-11-
00
w 02
Ow
the perturbed gradient with learning rate of a11 and a22, as shown in line 4.
3.4 The Noise Level
the previous section, we explained why Luain amounts to noise injection into the model. Next, we provide the intuition of why the proposed algo- rithm can detect the noise variance automatically. When introduce noise into the model, the train- ing loss Luain is expected to rise. The greater the amount of noise added, the larger the anticipated increase in Luainn In other words, Luain generally an increasing function of the noise level. Hence if we just minimize Luain, then the optimal noise variance would just be 0. The reason our algorithm can learn meaningful non-zero noise is due to the existence of the second term LPAC in the loss, which is decreasing function of the noise level when the noise converges towards the prior distri- bution. As a result, we expect that minimizing the total loss, Iiain + LPAC, will find us an optimal point for the noise level, and therefore achieve au- tomatic learning the noise. This is the basic idea of the proposed PAC-tuning algorithm that will be described in the next section.
(I((9.1(0L) + (L(()))
To force these KL divergences small for extremely large models, we leverage the PAC-Bayes bound proposed in Zhang et al. (2023), a variant of the basic PAC-Bayes bound J(0, P, described in Section 3.2. The final objective function want to minimize, omitting learnable parameters of the prior distribution variance, e.g., À and B, for sim- plicity, [(: C, w):
Lugin
e(D; = 1
(x,, De: 0,
(In J KL ((0) + (2 (og)
7K2
ym
Lesc
where { and e are the posterior distribution vari- ance associated with # and w respectively, {E,, 9:) 3)) is the training dataset, 5 E (0, 1) is the probability of failure, can be set to any value within a bounded [71, [71,72] specified by the users. and K K(X, B, 71 72) >> 0 is the effective variance of the training loss when the prior variances for (9, w) are set (A, 3). We refer readers to Section 4 of Zhang et al. (2023) for more details about ? and K. This objective function is obtained by mak- ing the following assumptions: (1) the prior distri- butions of the PLM classifier are 508 - NN AI) and Pa = N (wo, IT), where Og and wp are the initialized parameter weights, and (2) in each gra- dient update I, the posterior distributions of the PLM classifier are 2. O, + (N diag(E)) and 2 = w, (O, diag(c)) where 8 and wt are the current parameter weights for the gradient update step t.
After the training is complete. the learned noise levels be used for model interpreta- tion/validation, as they reflect how important each model parameter is the final performance. For example, a model parameter associated with large learned noise level is less important than one with a small noise level. More concretely, the trained model parameter is (1, 1, 1) and the learned noise level by PAC-training is (10, 1, 10), then it indi- cates that the second model parameter is more important than the first and the third because its associated noise injection level is low.
3.5 PAC-tuning
Previous work on PAC-Bayes training all targeted the one-time training of a neural network. fine-

---

each gradient update, we sample noise TI and 72 from a standard normal distribution and multiply them by the learned noise variance (E* and E" from Stage to replace line 3 of Algorithm as:
I( m)
Classification Layer
Classificetion Leyer e
Update
PLM
PLM
0
0 = 0 5
w = Ve T2
Input:tels entertoining story
Ingut:tels entertohnnng
Stage
Stage
4 Experiments and Analysis
Figure 1: PAC-tuning Pipeline. In Stage I, we update the model parameters and noise variance by minimizing the PAC-Bayes bound J described in Section 3.2. Then the optimal noise variance 5* and e" are learned after Ti training epochs. Next, fine-tuning continued in Stage 2 with Algorithm but the noise variance is fixed as &* and e", and the objective function is training loss Tumin only. Model parameters and updated during both fine-tuning stages.
In this section, we outline our experimental settings, dataset, and baseline models used for evaluation. Section 4.4 discusses the experimental findings. Section 4.5 concludes with an analysis of the sta- bility of our PAC-tuning approach.
4.1 Experimental Settings
We conduct extensive experiments with PAC- tuning and baseline PLMs over 5 text classifica- tion tasks of the GLUE benchmark as shown in Tables 1 and 2. We adopt the HuggingFace imple- mentations BERT and GPT-2 as backbone and add one fully-connected layer to be the classi- fication layer. To simulate a few-shot learning sce- nario, we randomly sample 100 instances from the original training set and use the whole development set to evaluate the classification performance. All experiments repeated 5 times and report the average performance over 5 seeds" on the original development set. All model architectures have the same hyperparameters and optimizers in all exper- iments, except the training epochs in PAC-tuning (as further detailed in Appendix A). We freeze the parameters associated with embeddings and do not update them during fine-tuning.
Figure shows the pipeline of our proposed PAC- tuming technique. The implementation contains stages. In Stage 1, by minimizing the objec- tive J over Ti epochs, the optimal noise variance <* and e* and the model parameters and w are updated. Afterward, we leverage PGD on the PLM (Algorithm 1) with fixed noise levels as 5 and e* to update 0 and w. Two stages of fine-tuning are re- quired because minimizing J cannot usually make the PLM classifier fit the downstream data very well due to the existence of the LPxc term. During stage 2, the LPAC term dropped; therefore the PLM classifier can fit the downstream data well.
The target of Stage is to estimate posterior vari- ance & and e as well as update model parameters?:
5","",0'00 ~
argmin C,, E.c0w.
For the implementation of PAC-tuning, set the learning rate for the variances associated with the PLM, & and X, to 0.1, and the learning rate for the variances of the classification layers, e and B3, to initialized as 0.5 and decreased to be 90% ev- ery 10 gradient updates, until the minimal of 0.01. We chose the loss interval 7 as 10 for the tasks of SST and CoLA, and used 5 for the remaining tasks. PAC-tuning Stage I runs for 250 epochs with a maximum training epoch of 300. However, the convergence of Stage I depends on the diffi culty of the considered task. For the SST task, a stage with 100 epochs can ensure convergence,
To reflect our greater confidence in 0 than w, we initialize & to be smaller than e. Meanwhile, we follow the convention to use smaller learning rate for the 0 than The small learning rate of 0 would in turn result in a smaller gradient for the corresponding noise &. Therefore, we need larger learning rate for g to neutralize this effect. In addition, dropout introduces extra noise to model parameters (Wei et al., 2020), resulting in a conflict with the noise injection by our proposed method. To effectively employ PAC-tuning, dropout must be disabled.
Given the learned posterior variance, Stage 2 continues fine-tuning 0 and through PGD. In
hhtps://gluebeenctaak.coomm
'htus://huugiingface http://innnnnnneeeeeebeeeeeee base-unnased
2 hhtps://huginngface co/gpt2
PPlcss note À and / are also optimized in Stage
'Sedds used: 1, 2, 10, 26, 100

---

but 250 epochs of Stage is enough for all of the experiments reported in this paper.
efficient while alleviating overfitting to down- stream tasks.
Prefix-tuning (Li and Liang, 2021; Liu et al., 2022) optimizes a sequence of continuous task-specific vectors added to the beginning of the input sequence, known as prefixes, while keeping the PLM parameters frozen. It pro- vides more efficient and effective approach for fine-tuning PLMs by optimizing these tinuous prefixes.
4.2 Dataset
Five tasks of the GLUE benchmark are used to val- idate our proposed fine-tuning method: the Corpus of Linguistic Acceptability (CoLA), the Stanford Sentiment Treebank (SST), a mixture of the two datasets MultiNLI Matched and MultiNLI Mis- matched (MNLI-m), Question NLI (QNLI, and Recognizing Textual Entailment (RTE).
BitFit (Ben Zaken et al., 2022) subnet- work fine-tuninn suue-tuning method that only optimizes the bias terms of the PLM. By targeting a specific subset of the model parameters, Bit- Fit achieves competitive performance by fine- tuning the entire model, and is especially ef- fective with smaller training datasets.
4.3
Baseline Methods
The following baseline methods represent current, typical approaches for fine-tuning.
Vanilla-tuning is the vanilla, basic parameter-
tuning without any add-on regularization. Data Augmentation is implemented in this work with Back Translatio (Sennrich et al., 2016) to control the quality of augmented data. Translanslt
4.4 Experimental Results
Tables and 2 show the experimental results of our proposed PAC-tuning approach compared to other fine-tuning methods when used with the two back- bone PLMs, BERT and GPT-2, respectively. The first column lists the specific fine-tuning method described in Section 4.3. The first three techniques, vanilla-tuning, data augmentation, and noise in- jection are instances of parameter-tuning methods. The next two techniques, LoRA and prefix-tuning, are examples of crrammter-efficient--ue next 5 columns correspond to each GLUE bench- mark task. Results for each task are reported in terms of accuracy, except for the CoLA task which uses the Matthew's correlation coefficient (MCC). The final column reports the average results for each fine-tuning approach across all 5 tasks.
model-based augmenta- tion method, which first translates a sequence of tokens into another language and then trans- lates it back to the original language. We mix the sampled training data and augmen- tation data together as the training set. For benchmarks with paired inputs, e.g., MNLI- QNLI, and RTE, generate augmented samples for each training sample. One is gen- erated from the first part of the input and the other is generated using the second part of that input. For the remaining benchmarks (SST and CoLA), we generated only one augmented sample using back translation.
The
Noise Injection (Orvieto et al., 2023) theo- retically and empirically proves that noise in- jection into randomly selected layer in each gradient update can avoid large loss variance and effectively implement explicit regulariza- tion to overparameterized models.
Overall, PAC-tuming achieves the best average performance with both PLMs, but is not the best fine-tuning approach for the MNLI-m task given the BERT-base backbone. The average perfor- mance of parameter-tuning methods is better than that of parameter-efficient tuning methods, though LoRA the second best fine-tuning method in Table 1. These experimental results show support- ive evidence for future research in applying PAC- tuning for fine-tuning PLMs for downstream tasks. When applied to the BERT backbone (Table 1), in the tasks of CoLA and SST. PAC-tuning's per- formance exceeds other fine-tuning baselines by a large margin. The performance gain for the QNLI and RTE tasks is somewhat smaller, but still signifi- cant. However, PAC-tuning is worse than data aug-
Low-Rank Adaptation (LooR) (Hu et al., 2022a) aims to address the challenges of fine-tuning PLMs by leveraging low-rank ap- proximations of the model's weight matrices, achieving more efficient adaptation to spe- cific tasks or domains. The low-rank adap- tation matrix amplifies important features for specific downstream tasks that were learned but not emphasized in the general pretrained model, making the adaptation process more

---

BERT
CoLA (MCC)
SST (Aceuracy)
MNLI-m (Accuracy)
QNLI (Accuracy)
RTE (Accuracy)
Average
Vanilla-tuning
233
.773
.369
.702
.589
.533
Data Augmentation
.171
8877
.395
.705
.994
.536
Noise Injection
233
.783
.311
.706
.588
.536
LoRA
.298
.792
.385
.669
.592
.547
Prefix-tuning
.191
.704
.375
649
.565
.497
BitFit
.267
.768
.376
.647
.588
.510
PAC-tuning
.335
.834
.387
.709
.601
.573
Table 1: Experimental Results for BERT BERT-base-uncee buss--uncseed Backbone Model. The best results are highlighted in bold. PAC-tuning outperforms other fine-tume fine-tuning methods in all tasks except MNLI-m, where data augmentation contributes to the best performance. PAC-tuning achieved the best average performance across all 5 tasks.
GPT-2
CoLA (MCC)
SST (Aceuracy)
MNLI-m (Accuracy)
QNLI (Accuracy)
RTE (Accuracy)
Average
Vanilla-tuning
048
.774
.353
.560
.572
.461
Data Augmentation
031
.777
.355
.573
.574
.462
Noise Injection
003
.713
.369
.550
.552
.451
LoRA
.033
.703
.366
.545
.500
.445
Prefix-tuning
.071
-521
.352
.523
.525
.398
BitFit
007
.586
.666
.542
.520
.432
PAC-tuning
.085
.815
.373
.576
.486
Table 2: Experimental Results for GPT-2 Backbone Model. The hest results are highlighted in bold. PAC-tuning outperforms all other fine-tuning methods.
mentation and parameter-efficien tuning methods for the MNLI-m task. Data augmentation- based fine-tuning is the best method for the task of MNLI- m and is the second or third best method for SST, QNLI, and RTE. However, it performs worse than vanilla-tuning for the CoLA task. It is the second best method in terms of stable performance across five tasks, indicating the effectiveness of data aug- mentation low-resource setting.
venting the use of PAC-Bayes training in practical settings. Our results of PAC-tuning demonstrate that PAC-Bayes training can be used with even very large models like PLMs, that have never been considered before.
4.5 Stability Analysis
The PAC-Bayes bound contains term relevant to data size and the KL--divergence term is associ- ated with model size. Therefore, we conduct thor- ough experiments to analyze how PAC-tuning' ss performance changes given different data sizes and model sizes. Table 3 shows the performance BERT-based fine-tuning methods with respect to training dataset size of 50 and 20. We construct the training dataset by implementing random sam- pling over the training set of SST and RTE. When training data size drops to 20, the performance of PAC-tuning worse than prefix-tuning by very small margin. Considering the test size RTE is small, the performance difference between prefix- tuning and PAC-tuning implies that both methods have very close generalization performances.
According to Table 2, the overall performance of GPT-2 is worse than that of BERT-based fine- tuning methods, particularly the task of CoLA. This is consistent with previous findings (Liu et al., 2021; Radford et al., 2019). However, the addi- tion of our method improves the fine-tuned perfor- mance, and our method is the best fine-tuning ap- proach for all tasks. All fine-tuning methods show similar trends in performance with the BERT backbone.
The overall good performance of PAC-tuning proves its feasibility and usefulness for fine-tuning PLMs for few-shot text classification tasks. This typical application scenario introduces two key challenges for applying PAC-Bayes training: larger model sizes and smaller data sizes, which are gen- erally considered to result vacuous bounds, pre-
Table 4 describes the classification results when considering fine-tuning methods on the SST and RTE tasks with DERT--aagg-unncseed
uncased as the back-

---

Training Size
RTE Training Size
20
Vanilla-tunin
.772
.601
Vanilla-tuning
.30
.577
Data Augmentation
.783
.885
Data Augmentation
.533
.507
LoRA
.756
.566
LoRA
.388
.144
Prefix-tuning
.672
.S72
Prefix-tuning
.538
.536
BitFit
.746
.610
BitFit
.543
.517
PAC-tuning
.810
.620
PAC-tuning
.546
.532
Table 3: Stability Analysis Training Dataset Sizes. This table presents the accuracy on development sets for the SST and tasks while varying training dataset sizes. PAC-tuning's performance drops as the CoLA data size decreases but still the best fine-tuning method given 50 training samples.
BERT-large
SST
RTE
Vanilla-tuning
.832
.561
Data Augmentation
836
.591
PAC-turning
LoRA
845
.560
2.0
Training with noise from Stage 1
Prefix-tuning
.721
.542
BitFit
8004
.546
PAC-tuning .848
.565
training loss
1.5
Table Stability Analysis Larger Model Architecture. PAC-tuning outperforms other fine-tuning methods the task SST when using BERTTlarge-uncased. The training data size is fixed at 100.
1.0
0.5
0.0
bone model. With this larger model, all fine-tuning methods have a performance increase over the two tasks. PAC-tuning is the best method in the SST task and the second best in the task of RTE, where data augmentation outperforms all methods. When viewed with the main experimental results of Sec- tion 4.4, these stability tests further validate the usefulness of leveraging PAC-Bayes training, via PAC-tuning, to fine-tune PLMs in the challenging settings of small training data availability and ex- tremely large pretrained models.
200
210
220
230
240
250
training
Figure 2: From the beginning of Stage 2, the noise learned in Stage I is applied fine-tune BERT-based model the SST dataset from scratch, as the blue line shows. Beginning the 200 epoch continue PAC-tuning, as shown in the red line, and leverage the learned noise in Stage fine-tune the model from scratch, described the blue line.
and validate how the training loss changes the fine-tuning process. From Figure 3, it is clear that the training loss stagnates around 1.5 at the end of Stage (200 epochs), which indicates that the model has not fit the data. This is because the existence of the LPAC term in the objective func- tion prevents the optimizer from further decreasing Luain- As long as Stage 2 starts, the model quickly fits the data and the training loss almost zero. More discussion on the two-stage training schema is available in Appendix B.
5
Discussion
5.1 The Role of Stage 1
Stage learns the noise variance to be used in Stage and prepares the model be a good initialization state for Stage 2. Figure indicates that if we start Stage 2 from the initial pretrained model and not from the learned model from Stage I, then the PGD steps in Stage cannot converge. This means both the level of the noise injection and the initialization learned from Stage are important for the success of Stage 2, showcasing the role of Stage for the PAC-tuning approach.
Advice for Applying PAC-tuning
In this section, we wish to share recommendations for using PAC-tuning, since the training process of PAC-tuning is different from conventional fine- tuning progress.
5.2 The Necessity of Stage 2
Stage 1, the target is to minimize Tarain Leac which is larger than the training loss alone. Therefore, users may not observe a
To empirically verify the necessity of Stuge 2 in PAC-tuning, we run PAC-tuning on the SST dataset

---

generally considered be the two main obstacles for improving generalization through PAC-Bayes training. With extensive experiments on 5 GLUE benchmark tasks, we observed that the performance of PAC-tuning is competitive to other fine-tuning methods and more stable with respect to differ- ent model sizes and training dataset sizes. PAC- tuning, our proposed pretrained language model fine-tuning method,
10
training loss
be expanded in several ways for future work. More larger-sized models should be validated to fully explore the effective- ness of PAC-tuning, would also be interesting to augment other fine-tuning techniques with PAC- tuning, especially data augmentation. Lastly, the performance of PAC-tuning is largely determined by the convergence of Stage 1, necessitating more studies determine how to make Stage 1 con- verge quickly and robustly. Our experimental re- sults demonstrate the usefulness of PAC-tuning and the potential to consider NLP problems from the point-of-view of generalization, a less explored PLM-optimizzation approach in the NL. commu- nity.
2
Stage 2 starts
0
150
200
250
training epoch
Figure 3: The training trajectory w.r.t the training loss (cross- entropy loss) in the course of PAC-tuning with the SST dataset and BERT-base We take 200 training epochs for Stage I, and Stage starts from the 2h0 epoch as indicated by the black vertical line.
large decrease in the training loss. As long as the total loss is reducing, PAC-tuming is progressing correctly.
Limitations
Since the variances of the noises are non- negative, we used exp(2p) to model them, where p is a trainable parameter. In addi- tion, we initialize the standard deviation using the initial weights. More specifically, p is initialized as the log magnitude of the initial weights
Although we empirically validated the effective- ness of our proposed PAC-tuning method, there is still room for improvement. In particular, we cannot validate how PAC-tuming can be improved with the full-batch gradient update due GPU hardware access limitations. Related to this, we did not perform an exhaustive best hyperparameter search, and instead defaulted to conventional learn- ing rates and batch sizes to ensure fairness across all experiments.
An effective way check the status of Stage is check the mean value of the posterior variance. If the mean value does not change, users should increase their lcarning rate or increase the learning rate of the PLM.
It is also possible that the reported performances may not be the best performances. BERT and GPT- are not the newest language models and they are small compared to currently popular large language models. Therefore, more experiments for larger models are required, including experiments with close-sourced yet powerful models such as GPT-4. Furthermore, our experiments should be repeated in order to compare the performance of PAC-tuning and prompt-based techniques for validation against models such as ChatGPT and BARD.
The prior variance parameters can easily ap- proach convergence and are less sensitive to learning rate. Therefore, users can begin with large learning rate and decrease it gradually. For the learning rate of the posterior variance, since the gradient very small, we recom- mend readers to pick up a large learning rate.
7 Conclusions and Future Work
In this paper, propose PLM fine-tuning method, PAC-tuning, for few-shot text classifica- tion. PAC-tuning is based on PAC-Bayes train- ing and perturbed gradient descent. We leverage PAC-tuning in the more challenging settings of larger models and smaller training data, which are
Acknowledgement
This work is supported in part by the National Sci- ence Foundation (NSF) grant CCF-2212065.

---

References
Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. 2020. Sharpness-awer aware mini- mization for efficiently improving generalization. In International Conference on Learning Representa- lions.
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2020. Better fine-tuning by reducing representational collapse. In International Conference on Learning Representations.
Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In International Conference on Learning Representations.
Farid Arthaud, Rachel Bawden, and Alexandra Birch. 2021. Few-shot learning through contextual data augmentation. In Proceedings of the 16th Confer- ence of the European Chapter of the Association for Compulational Linguistics: Main Volume, pages 1049-1062.
Tianyu Gao, Adam Fisch, and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Anmual Meet- ing of the Association for Computational Linguistics and the 1ith International Joint Conference Nutu- ral Language Processing (Volume 1: Long Papers), pages 3816-3830.
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022. BitFit: Simple parameter-efficient fine-tuning for trande masked longuage-models. In Proceedings of the 60th Annual Meeting of the As- sociation for Computational Linguistics ( (Volume 2: Short Papers), pages 1-9, Dublin, Ireland. Associa- tion for Computational Linguistics.
Yuxian Gu, Han, Zhiyuan Liu, and Minlie Huang. 2022. Ppt: Pre-trained prompt tuning for few-shot learning. In Proceedings of the 60th Anmual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers). pages 8410-8423.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2020. Supervised contrastive learning for pre-trained language model fine-tuning. In Interna- tional Conference on Learning Representations.
Fengxiang He, Tongliang Liu, and Dacheng Tao. 2019. Control batch size and learning rate to generalize well: Theoretical and empirical evidence. Advances in Neural Information Processing Systems, 32.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understand- ing. arXiv preprint arXiv:1810.04805
Tianxing He, Jun Liu, Kyunghyun Cho, Myle Ott, Bing Liu, James Glass, and Fuchun Peng. 2021. Analyzing the forgetting problem in pretrain-finetuning of open- domain dialogue response models. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, pages 1121-1133
Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannanch Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight ini- tializations, data orders, and early stopping. arXiv preprint arXiv:2002.06300
Gintare Karolina Dziugaite and Daniel M Roy. 2017a. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data. arXiv preprint 100.:
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li. Shean Wang. Lu Wang. and Weizhu Chen. 2022a. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
Gintare Karolina Dziugaite and Daniel M Roy. 2017b. Computing nonvacuous generalization bounds for deep (stochastic) neural networks with many more parameters than training data.
Shengding Hu, Ning Ding, Huadong Wang, Zhiyuan Liu, Jingang Wang, Juanzi Li, Wei Wu, and Maosong Sun. 2022b. Knowledgeable prompt-tuning: Incor- porating knowledge into prompt verbalizer for text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 2225 -2240.
Steven Y Feng, Varun Gangal, Jason Wei, Sarath Chan- dar. Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. A survey of data augmentation approaches for nlp. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 968 -888.
Haoming Jiang, Pengcheng He, Weizhu Chen, Xi- aodong Liu, Jianfeng Gao. and Tuo Zhao. 2020. Smart: Robust and efficient fine-tur the-tunnng for pre- trained natural language models through principled regularized optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2 2177-2190.
Andrew Foong, Wessel Bruinsma, David Burt, and Richard Turner. 2021. How tight can pac-bayes be in the small data regime? Advances in Neural Informa- fion Processing Systems, 34:4093-4105.

---

Yiding Jiang, Behnam Neyshabur, Hossein Mobahi, Dilip Krishnan, and Samy Bengio. 2019. Fantas- tic generalization measures and where to find them. In International Conference on Learning Representa- lions.
Marius Mosbach, Maksym Andriushchenko, and Diet- rich Klakow. 2020. On the stability of fine-tuning bert: Misconceptions, explanations, and strong base- lines. In Internationall Conference on Learning Rep- resentations.
Chi Jin, Praneeth Netrapalli, Rong Ge, Sham M Kakade, and Michael I Jordan. 2021. On nonconvex opti- mization for machine learning: Gradients, stochastic- ity. and saddle points. Journal of the ACM (JACM). 68(2):1-22 1-29.
Antonio Orvieto, Hans Kersting, Frank Proske, Francis Bach, and Aurelien Lucchi. 2022. Anticorrelated noise injection for improved generalization. arXiv preprint arXiv:2202.02831/
Antonio Orvieto, Anant Raj, Hans Kersting, and Fran- cis Bach. 2023. Explicit regularization over- parametrized models via noise injection. In Inter- national Conference on Artificial Intelligence and Statistics, pages 7265-7287. PMLR.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- nieszka Grabska-Barwe Barwinska, et al. 2017. Over- coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13)3321-33526
Alee Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9.
Varun Kumar, Hadrien Glaude, Cyprien de Lichy, and Wlliam Campbell. 2019. A closer look at feature space data augmentation for few-shot intent classi- fication. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (Deepla 2019), pages 1-10.
Omar Rivasplata, Vikram M Tankasali, and Csaba Szepesvári. 2019. Pac-bayes with backprop. arXiv preprint arXiv: 1908.07380.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine uranslation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86-96.
Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. 2019. Mixout: Effective regularization finetune large-scale pretrained language models. In Interna- tional Conference on Learning Representations.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pages 4582- 4597.
Colin Wei, Sham Kakade, and Tengyu Ma. 2020. The implicit and explicit regularization effects of dropout. In International conference on machine learning. pages 10181-10192. PMLR.
Xing Wu, Chaochen Gao, Meng Lin, Liangjun Zang. and Songlin Hu. 2022. Text smoothing: Enhance var- ious data augmentation methods on text classification tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Vol- ume 2: Short Papers), pages 871-875.
Yuanzhi Li, Colin Wei, and Tengyu Ma. 2019. Towards explaining the regularization effect of initial large learning rate in training neural networks. Advances in Neural Information Processing Systems, 32.
Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan. Baobao Chang, Songfang Huang, and Fei Huang. 2021. Raise a child large language model: To- wards effective and generalizable fine-tuning. In Pro- ceedings of the 2021 Conference on Empirical Meth- ods in Natural Language Processing, pages 95 14 9528.
Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. 2022. P P-tuning: fine-tuning
Prompt tuning can be comparable across scales and tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers). pages 61-68, Dublin, Ireland. Association for Computational Lin- guistics.
Yue Yu, Simiao Zuo, Haoming Jiang. Wendi Ren, Tuo Zhao, and Chao Zhang. 2021. Fine-tuning pre- trained language model with weak supervision: A contrastive-regullaized self-training approach. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies. pages 1063-1077.
Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2021. Gpt understands, too. arXiv preprint arXiv:2103. 10385.
Andreas Maurer. 2004. A note the pac bayesian theorem. arXiv preprint cs/041 1099.
David A McAllester. 1998. Some pac-bayesian theo- rems. In Proceedings of the eleventh annual confer- ence on Computational learning theory. pages 230- 234.
Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Wein- berger, and Yoav Artzi. 2020. Revisiting few-sample bert fine-tuning. International Conference on Learning Representations.

---

Tianyi Zhang, Felix Wu, Arzoo Katiyar. Kilian Q Wein- berger, and Yoav Artzi. 2021. Revisiting few-sample [bert] fine-tuning. In International Conference Learning Representations.
that learns the model posterior. Despite the use of two stages, as presented in Dziugaite and Roy (2017b), the method still faces challenges when dealing with unbounded loss functions, such as the commonly used cross-entropy loss in text classifi- cation tasks. Moreover, it demands a significant amount of training time.
Xitong Zhang, Avrajit Ghosh, Guangliang Liu, and Ron- grong Wang. 2023. Unlocking tuning -ree generaliza- tion: Minimizing the pac- pac-bayes bound with trainable priors. arXiv preprint arXiv:2305. 19243.
To the best of our knowledge, prior to this work, there has been no PAC-Bayes training method that outperforms the baseline methods on any popular task, especially when targeting complex architec- tures like transformers.
Jing Zhou, Yanan Zheng, Jie Tang. Li Jian, and Zhilin Yang. 2022. Flipda: Effective and robust data aug- mentation for few-shot learning, In Proceedings of the 60th Annual Meeting the Association for Compulational Linguistics (Volume I: Long Papers), pages 6646-8665.
The primary reason for the extended training epochs required in Stage of PAC-tuning is the necessity to effectively learn both the model and the noise. It is worth noting that all PAC-Bayes training methods that optimize these aspects also tend to require more training epochs. While this does result in longer running times, the benefit of learned noise is significant, as can be used enhance model calibration and support pruning.
Mo Zhou, Tianyi Liu, Yan Li, Dachao Lin, Enlu Zhou, and Tuo Zhao. 2019. Toward understanding the im- portance of noise in training neural networks. In ICML.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Gold- stein, and Jingjing Liu. 2019. Freelb: Enhanced ad- versarial training for natural language understanding. In International Conference on Learning Representa- lions.
A Hyperparameters
Hyperparameters
Setting
Optimizer
AdamW
Adam 31
0.9
Adam (3
0.98
Adam
le-3
Learning rate for
5e-5
Learning rate for w
le-2
Maximum training epochs
35
Weight decay
0.01
Batch size
32
Table 5: Hyperparameter Settings for the AdamW Opti- mizer.
B Two-stage Approach
Most PAC-Bayes training methods typically rely on a single-stage approach. However, these meth- ods are limited in their applicability, as they can only effectively handle bounded loss functions and shallow networks. They also struggle to optimize the noise prior, resulting in suboptimal final perfor- mance. We know of one other paper that used two stages but in different way. More explicitly, Dziu- gaite and Roy (2017b) introduce two-stage train- ing process, where the first stage focuses on leamn ing the model prior, followed by a second stage


Reviews:
[{"review_id": "lmYjqGN6J5", "reviewer": "Reviewer_NWbh", "summary": "", "questions": "Is the proposed method related to Bayesian neural networks (BNNs)? In BNNs, we also model the parameter as a distribution. The PGD part  is very similar to BNNs which is inferred using stochastic gradient langevin dynamics. Given this, I am very curious about whether the proposed method can improve uncertainty estimation and model calibration?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the great comments from the reviewer.  Here are our responses to the proposed questions.\n\n**Q: The chosen baselines in the experimental section appear somewhat unconvincing. Many are geared towards efficiency rather than improved generalization performance. Numerous fine-tuning techniques have been developed with an emphasis on bolstering generalization performance. How does the PAC training method stack up against the likes of SMART?**\n\n**A:** We did implement tests using SMART. Following the suggested hyperparameter settings, SMART performed lower than our vanilla-tuning baseline for most tasks. One possible cause could be the limited training dataset size hindering the search for adversarial samples, resulting in a less effective decision boundary of smoothness. Because of this, we did not report our SMART results, however we can add them into the final version with this explanation. Data augmentation is a widely used method for improving generalization and robustness by introducing more samples with off-the-shelf tools. Noise injection is considered a regularization over the trace of hessian matrix, improving generalization.\n\n**Q: Is the proposed method related to Bayesian neural networks (BNNs)? In BNNs, we also model the parameter as a distribution. The PGD part is very similar to BNNs which is inferred using stochastic gradient langevin dynamics. Given this, I am very curious about whether the proposed method can improve uncertainty estimation and model calibration?**\n\n**A:** As you correctly mentioned, BNN and PAC-bayes training both model the parameter as a distribution and they both target on mitigating overfitting and improving generalization. The differences are 1) PAC-Bayes training directly minimizes the generalization error while the relation between generalization the BNN minimizer is a bit vague 2) During training, BNN uses a fixed prior, while our PAC-bayes training algorithm optimizes over a set of priors for the best one, therefore is tighter 3) During the test stage, our PAC-bayes training algorithm uses a fast deterministic predictor while BNN algorithms use probabilistic predictors.  These three differences all contribute to the reported improvement in the numerical results. For your  second question, yes, since PAC-Bayes training also learns the distribution, we do expect to apply it to uncertainty estimation and model calibration in  our future work.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper is cogently composed with a lucidly articulated motivation.\n2. The proposed methodology is technically robust. Employing PGD to minimize the PAC-Bayes bound emerges as an astute and efficacious approach.", "reasons_to_reject": "The chosen baselines in the experimental section appear somewhat unconvincing. Many are geared towards efficiency rather than improved generalization performance. Numerous fine-tuning techniques have been developed with an emphasis on bolstering generalization performance. How does the PAC training method stack up against the likes of SMART?\n\nSMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization.", "questions_to_authors": "Is the proposed method related to Bayesian neural networks (BNNs)? In BNNs, we also model the parameter as a distribution. The PGD part  is very similar to BNNs which is inferred using stochastic gradient langevin dynamics. Given this, I am very curious about whether the proposed method can improve uncertainty estimation and model calibration?", "paper_topic_and_main_contributions": "This paper introduces a novel fine-tuning technique for pre-trained language models (PLM) aimed at enhancing generalization performance. Termed \"PAC training,\" the method seeks to directly minimize the PAC-Bayes generalization bound and is executed in two distinct stages. Initially, the method determines the optimal parameter distribution variances. Subsequently, the gradient is adjusted by injecting noise, leveraging the variance ascertained in the first phase. Through rigorous experimentation, the paper demonstrates that PAC tuning surpasses existing fine-tuning methods in terms of generalization performance.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7GvfGAs9Lz", "reviewer": "Reviewer_msWo", "summary": "", "questions": "Please respond to the concerns in weaknesses.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thanks for the reviewer's comments. \n\n**Q: Although the proposed two-stage finetuning method brings clear improvements compared to previous tuning methods, I'm curious about the performance of the tuned model after the first stage. And it's better to explain further why the two-stage design is indispensable instead of a collaborative optimization.**\n\n**A:** In conclusion stage 1 is necessary to determine the learned noise and help the model to be in a good initialization point for continual fine-tuning. Stage 2 is necessary to minimize training loss as much as possible while avoiding under- and overfitting. This [figure](https://www.dropbox.com/scl/fi/kar9kw8xmmg2lieqvwwel/ce_loss.png?rlkey=ogt4ght9mtimqjkyhxp027hsg&dl=0) shows that the model underfits training data before stage 1, and well fit the training data during stage 2. To understand the effect on the model initialization point, this [figure](https://www.dropbox.com/scl/fi/dowt6xx96ck03zjirqzlw/trainimprovestage1.png?rlkey=l00o4qqmve1mye6pwoufrvwsw&dl=0) shows that if we directly inject the learned noise into the pretrained model without stage 1, the training loss diverges. It would be very great if we can do collaborative optimization over the model and noise, the underfitting in stage 1 is inevitable because of the existence of the PAC loss.\n\n**Q: It is advisable to provide a comparison of the number of tunable parameters and the amount of tuning time budgets between the listed tuning methods in the experiment.**\n\n**A:** One of the benefits of the proposed algorithm is easy tuning.  In contrast to the Adam baseline, the final performance of the proposed method depends very little on the hyper-parameters.  In particular, the proposed method does not need dropout, and the generalization is not affected much by varying the learning rates, as long as the learning rate is not too large to cause it to diverge. Likewise,  for the proposed method, using large batch sizes or small batch sizes  yield similar generalization result.  Having said that, if one hopes to shorten the computational time, then some tuning may be needed on the learning rate, as a large learning rate grants a faster training convergence.  But in the future we plan to use the method  [1] to automatically tune the learning rate.We would  add a short  discussion on this in the final version.\nWe have provided the table below to answer your question, and will include this in the final version. \nIn  the table below, we  compare the running time with  those of vanilla-tuning, data augmentation, and noise injection on the SST dataset. Since  parameter-efficient approaches are not competitive to those methods, we did not report the running time for parameter-efficient approaches.  All experiments (except PCA) take 15 training epochs over 5 seeds, and we run PAC-tuning for  280 epochs (we can further reduce the epochs for Stage 1 for most of the tasks). Though PAC-tuning uses more epochs, we just need to run it once, as opposed to having to run multiple times of other methods for hyper-parameter tuning. Therefore, the running times of all the methods are comparable. \n\n\n\n\n| Method            | Hyper-parameter setting                                                                                          | Running time  |\n|-------------------|------------------------------------------------------------------------------------------------------------------|---------------|\n| Vanilla-tuning    | batch_sizes: [8, 16, 32, 64] Learning rate:[1e-5, 2e-5, 3e-5, 5e-5]                                              | 19.8 minutes  |\n| Data augmentation | batch_sizes: [8, 16, 32, 64] Learning rate:[1e-5, 2e-5, 3e-5, 5e-5]                                              | 38.8 minutes  |\n| Noise Injection   | batch_sizes: [8, 16, 32, 64] Learning rate:[1e-5, 2e-5, 3e-5, 5e-5] noise_level = [1e-8, 1e-7, 1e-6, 1e-5, 1e-3] | 26.8minutes   |\n| PAC-tuning        | Batch size: [32] Learning rate: [5e-5] Learning rate for posterior noise variance:[0.01]                         | 35.25 minutes |\n\n**Q: The foreshadowing of introducing PAC training is poor. It's better to explain the minimization of the PAC-Bayes bound solves what problem in the current fine-tuning landscape.**\n\n**A:** Upon acceptance, we will add more details about how minimizing pac-bayes bound helps to minimize the generalization/test error   on the fine-tuning task.  \n\n**Q: The current experiment only involves the NLU task, and testing on the NLG task would be useful.**\n\n**A:** We feel there is no technical obstacle in applying the proposed method on NLG.  However, good generalization is only one of the desired properties for NLG. High-level evaluation metrics like coherence, harmless or model-based evaluation metrics such as Bert-score are also very important. How to incorporate non-generalization evaluation into the PAC-Bayes training is one of our feature directions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "PAC-tuning presented in this paper takes the first step to applying the PAC training to LM fine-tuning. It reveals an avenue of controllably tuning the large pre-trained LMs with few-shot samples. And the reported improvements on standard benchmarks compared to existing model tuning approaches are remarkable.", "reasons_to_reject": "Weaknesses:\n\nA. Although the proposed two-stage finetuning method brings clear improvements compared to previous tuning methods, I'm curious about the performance of the tuned model after the first stage. And it's better to explain further why the two-stage design is indispensable instead of a collaborative optimization.\n\nB. It is advisable to provide a comparison of the number of tunable parameters and the amount of tuning time budgets between the listed tuning methods in the experiment.\n\nC. The foreshadowing of introducing PAC training is poor. It's better to explain the minimization of the PAC-Bayes bound solves what problem in the current fine-tuning landscape.\n\nD. The current experiment only involves the NLU task, and testing on the NLG task would be useful.", "questions_to_authors": "Please respond to the concerns in weaknesses.", "paper_topic_and_main_contributions": "This paper presents a novel few-shot finetuning technique for pre-trained LMs. They introduce the PAC training strategy to model fine-tuning and obtain a remarkable improvement compared to previous fine-tuning approaches.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YILkxOIwpN", "reviewer": "Reviewer_6Piy", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the helpful comments. Below our responses to the concerns above.\n\n**Q: In-depth analysis of the stability, convergence, and other performance aspects of PAC-tuning, along with a more comprehensive comparison with other fine-tuning methods, should be provided.**\n\n**A:** Compared to traditional training, our proposed PAC-tuning approach utilizes a different objective function and an extra parameter to model the noise level. But the optimization algorithm we used to solve this new optimization problem is still Adam, therefore, the convergence property and stability follow those of Adam’s directly.  We will add the training trajectory of PAC-tuning and other methods for more comprehensive comparison.\n\n\n**Q: It is necessary to investigate whether the two-stage approach of PAC-tuning introduces additional training time and computational overhead. Furthermore, a more extensive comparison with other methods should be included.**\n\n**A:** The first stage in the two-stage approach indeed increases the running time for a single run. But since it avoids hyper-parameter tuning, the overall running time of the PAC-tuning is similar to other methods as shown in the table in our response to the reviewer **msWo**. Even though the PAC-tuning introduces the noise level as extra parameters, we still only need one back-propagation in each iteration, as the result of this one back-propagation can be used to compute the gradients of both the model and the noise. Although we implement stage 1 with 250 epoch for all tasks in our reported experiments, for some tasks the training epochs for stage 1 can be very small such as sentiment analysis, due to the less difficulty of this task.if one hopes to shorten the computational time, then some tuning may be needed on the learning rate, as a large learning rate grants a faster training convergence.  But in the future we plan to use the method  [1] to automatically tune the learning rate. We would  add a short  discussion on this in the final version.\n\n\n[1] Defazio, A., & Mishchenko, K. (2023). Learning-rate-free learning by D-adaptation. arXiv preprint arXiv:2301.07733.\n\n**Q: More details regarding the implementation of PAC-tuning and the exploration of key hyperparameter settings should be provided.**\n\n**A:** We will open-source our code and add more details about how to compute hyperparameters, e.g. K,  in section 3.5, upon acceptance. \n\n**Q:  The computational complexity and the curse of dimensionality associated with PAC-Bayes bounds should be discussed. PAC-tuning currently considers only the parameter confidence differences between the pre-trained language model (PLM) and the classification layer, neglecting differences among different layers. The discussion could be expanded to include: 1) Confidence differences among different layers of the PLM, and 2) The computation complexity of PAC-Bayes bounds and the impact of the curse of dimensionality, along with potential mitigation strategies.**\n\n**A:** (1) Regarding the confidence with respect to different layers among the pre-trained part or the classification part: we did layer-wise prior parametrization and elementwise posteriror parameterization，which means, difference between layers are taken into consideration. We will add these details in the final version.  (2) The PAC-turning algorithm is almost as expensive as the traditional training. Even though the PAC-tuning introduces the noise level as extra parameters, we still only need one back-propagation in each iteration, as the result of this one back-propagation can be used to compute the gradients of both the model and the noise levels.  Therefore, the complexity of the optimization algorithm does not suffer from the curse of dimensionality. From our experiment, the PAC-Bayes bound also does not seem to suffer from the curse of dimensionality in the sense that the numerical value of the bound does not seem to grow with the number of model parameters.  This would be an interesting question to study. \n\n\n**Q: A comparison with current state-of-the-art prompt-tuning methods is lacking. If GPT-3 and GPT-4 are considered too powerful as baselines, alternative language models of different scales, such as BERT-large or DistilBERT, should be used to validate the scalability of the proposed method. The effectiveness of PAC-tuning needs further verification beyond experiments conducted solely on BERT-base.**\n\n**A:** The Prefix-tuning baseline in our paper is the referenced baseline methods of prompt-tuning with Bert-base/large and GPT-2.  We reported the results of PAC-tuning on BERT-large in Table 4.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "• Thorough and comprehensive content: The paper provides detailed explanations of the inspiration, background theory, method description, and experimental design of PAC-tuning. The overall structure of the paper exhibits clear and coherent reasoning.\n\n• Strong practicality of the method: PAC-tuning can be easily applied to existing pre-trained language models. Experimental results demonstrate that PAC-tuning achieves superior performance compared to conventional fine-tuning and various other enhancement methods.\n\n• Experimental results on the GLUE benchmark dataset show that PAC-tuning outperforms other methods, highlighting its effectiveness.", "reasons_to_reject": "• In-depth analysis of the stability, convergence, and other performance aspects of PAC-tuning, along with a more comprehensive comparison with other fine-tuning methods, should be provided.\n\n• It is necessary to investigate whether the two-stage approach of PAC-tuning introduces additional training time and computational overhead. Furthermore, a more extensive comparison with other methods should be included.\n\n• More details regarding the implementation of PAC-tuning and the exploration of key hyperparameter settings should be provided.\n\n• The computational complexity and the curse of dimensionality associated with PAC-Bayes bounds should be discussed. PAC-tuning currently considers only the parameter confidence differences between the pre-trained language model (PLM) and the classification layer, neglecting differences among different layers. The discussion could be expanded to include: 1) Confidence differences among different layers of the PLM, and 2) The computation complexity of PAC-Bayes bounds and the impact of the curse of dimensionality, along with potential mitigation strategies.\n\n• A comparison with current state-of-the-art prompt-tuning methods is lacking. If GPT-3 and GPT-4 are considered too powerful as baselines, alternative language models of different scales, such as BERT-large or DistilBERT, should be used to validate the scalability of the proposed method. The effectiveness of PAC-tuning needs further verification beyond experiments conducted solely on BERT-base.", "questions_to_authors": "", "paper_topic_and_main_contributions": "In this study, the authors present a novel two-stage fine-tuning approach called PAC-tuning, aiming to enhance the generalization performance of downstream NLP tasks. In the first stage of PAC-tuning, the authors propose to learn the noise variance and update the model parameters by minimizing the PAC-Bayes bound. In the second stage, the noise variance learned in the first stage is utilized for noise injection to further fine-tune the model. The experimental results on the GLUE benchmark dataset demonstrate promising performance, particularly in small-sample text classification tasks. The paper is rich in content, well-supported by comprehensive experiments, and presents clear arguments. However, it lacks an in-depth analysis of the stability and convergence properties of PAC-tuning, as well as a more comprehensive comparison with other existing methods.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "VaPTEbjT6f", "reviewer": "Reviewer_uBJv", "summary": "", "questions": "- How do the parameter updates differ for each stage? \n- How is each phase influencing the fine-tuning process? It would be nice to include the progress of the different losses during fine-tuning to help validate some claims.\n- Did you consider evaluating PAC-tuning without the few-shot component? Would you still require more training than existing fine-tuning methods?", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the comments and feedback from the reviewer.\n\n**Q: The method requires a lengthy extra stage in the fine-tuning process instead of directly fitting the PLM classifier to the downstream task. Even though this might be okay in specific scenarios, it is still a major bottleneck in the method's practicality. I would like to see the authors attempt at fixing this problem. For example, an ablation study where Phase 2 uses random SGD perturbations is important to see the importance of Phase 1 (or how it can be reduced).**\n\n**A:** Our method does require extra training epochs as well as more running time, as shown in the table of our response to the reviewer **msWo**. Though PAC-tuning needs extra epochs, our method increases downstream performance with minimal overhead. And the test time of PAC-tuning is as less as the vallina-tuning. The extra training epochs is relevant to the learning rate of posterior noise variance, one of our future directions is to achieve adaptive learning rate by referring to [1]. On the other hand, the general PGD methods learn scaler noise but PAC-tuning generate parameter-wise noises that are informative to measure the generalization-wise importance.\n\n[1] Defazio, A., & Mishchenko, K. (2023). Learning-rate-free learning by D-adaptation. arXiv preprint arXiv:2301.07733.\n\n**Q: There are minimal to no discussions about the hyperparameter ranges tested and how the values for ξ and λ used in the experiments are obtained.**\n\n**A:** Regarding the initialization of noise variance parameters, the posterior noise variance is obtained with pre-trained weights’ mean values as described in Section 5. The prior noise variance is initialized as the mean of the posterior noise variance, and we will add this detail to the final version.\n\n**Q: Since the results on the 5 GLUE tasks are sometimes contradictory regarding the best method, extending the number of tasks should be considered to avoid noisy conclusions. The authors should also consider using SuperGLUE instead of GLUE.**\n\n**A:** Most baselines take the GLUE benchmark, allowing us to make comparisons. SuperGLUE is more suitable for more powerful language models, this was a better pairing considering we are using BERT.\n\n**Q: The GPT-2 GLUE scores are extremely low (less than 50%) which indicates that the problem setting was not well thought of. E.g. GPT-2 might not be a suitable model to perform fine-tuning in a few-shot learning setting.**\n\n**A:** GPT-family of models are all about few-shot learning, our results show their weaker performance in few-shot classification tasks. Given the weaker performance, PAC-tuning can still improve generalization, showing the effectiveness of our proposed method across modal architects.\n\n**Q: Finally, there are no results exploring the incorporation of PAC-tuning with other fine-tuning techniques such as pruning and data augmentation.**\n\n**A:** Our method can be easily extended to incorporate other off-the-shelf fine-tuning techniques. However, data augmentation would not be a good addition since it may introduce OoD samples contradicting the IID assumption in the PAC-bayes bound, since unseen words would be introduced with popular augmentation methods such as EDA and back-translation. We initially explored this addition for our paper and empirically validated this drawback. The only data augmentation method that meets the IID assumption is Mix-up, but augmentation with Mix-up has been shown to be problematic in determining correct labels. In the vision domain, PAC-tuning beyond Mix-up would be a good idea due to the continuous nature of imaging data. Lastly, the learned noise level in PAC-tuning is an indicator for parameter importance w.r.t generalization, and can be used for pruning. Our method can be seen as a counterpart to pruning.\n\n**Q: How do the parameter updates differ for each stage?**\n\n**A:** Figure 1 in our draft shows how the parameters in our method are updated. The parameters of the prior and posterior noise variances are learned in Stage 1, and the pre-trained and classification layers are updated in both Stage 1 and Stage 2 with the same learning rate.\n\n**Q: How is each phase influencing the fine-tuning process? It would be nice to include the progress of the different losses during fine-tuning to help validate some claims.**\n\n**A:** Section 3.2 describes this progress. We take the sentiment analysis task as an example and visualize the status of both cross-entropy loss (L_train term in Section 3.2) and the KL loss (L_pac term in Section 3.2). We apply 200 epochs for Stage 1. Please find the descriptions of Section 3.2 visualized in the [figure](https://www.dropbox.com/scl/fi/kar9kw8xmmg2lieqvwwel/ce_loss.png?rlkey=ogt4ght9mtimqjkyhxp027hsg&dl=0) for cross-entropy loss with and the [figure](https://www.dropbox.com/scl/fi/of6u9f5gk25hs7yk63vd5/kl_loss.png?rlkey=r9om4pxt8cy345d5g27ft6oik&dl=0) for KL loss  \n\n**Q: Did you consider evaluating PAC-tuning without the few-shot component? Would you still require more training than existing fine-tuning methods?**\n\n**A:**  PAC-tuning works very well given a large number of training samples. Theoretically, a larger dataset size can make the L_pac loss term smaller, helping to minimize the L_train term and making the model fits training data better than that fine-tuned with only a few samples. We will add this in our final version as an analysis to the PAC-tuning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper focuses on exploring additional incorporations of PAC-based learning in deep neural networks, particularly large language models, which is a timely problem to tackle.", "reasons_to_reject": "- The method requires a lengthy extra stage in the fine-tuning process instead of directly fitting the PLM classifier to the downstream task. Even though this might be okay in specific scenarios, it is still a major bottleneck in the method's practicality. I would like to see the authors attempt at fixing this problem. For example, an ablation study where Phase 2 uses random SGD perturbations is important to see the importance of  Phase 1 (or how it can be reduced).\n- There are minimal to no discussions about the hyperparameter ranges tested and how the values for ξ and λ used in the experiments are obtained.\n- Since the results on the 5 GLUE tasks are sometimes contradictory regarding the best method, extending the number of tasks should be considered to avoid noisy conclusions. The authors should also consider using SuperGLUE instead of GLUE.\n- The GPT-2 GLUE scores are extremely low (less than 50%) which indicates that the problem setting was not well thought of. E.g. GPT-2 might not be a suitable model to perform fine-tuning in a few-shot learning setting.\n- Finally, there are no results exploring the incorporation of PAC-tuning with other fine-tuning techniques such as pruning and data augmentation.", "questions_to_authors": "- How do the parameter updates differ for each stage? \n- How is each phase influencing the fine-tuning process? It would be nice to include the progress of the different losses during fine-tuning to help validate some claims.\n- Did you consider evaluating PAC-tuning without the few-shot component? Would you still require more training than existing fine-tuning methods?", "paper_topic_and_main_contributions": "This paper introduces PAC-tuning, a two-stage fine-tuning process for pre-trained language models. The authors focus on few-shot fine-tuning to showcase the applicability of PAC-based learning in challenging scenarios. The method shows competitive results on 5 tasks of the GLUE benchmark, performed using BERT and GPT-2.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Missing year in Haojie et al.", "Typos_Grammar_Style_and_Presentation_Improvements": "- Section 3.2,. -> line 315\n- Several misuses of \\citet vs \\cite"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The author propose a hypeparameter optimization + gradient descent modification method which is inspired by minimizing a generalization bound. All reviewers agree that the synthesis of PAC-bayes generalization bounds with PLMs is interesting, and while there are some shared concerns about baselines and evaluations (to other regularized fine tuning methods, prompting etc), this is not a major issue.
Decision: Accept-Main