Representation Projection Invariance Mitigates Representation Collapse
Authors: Anastasia Razdaibiedina
Ashish Khetan
Zohar Karnin
Daniel Khashabi
Vivek Madan
Venue: EMNLP 2023 Findings (2023)
Keywords: representation learning
generalization
representation collapse
TL;DR: We propose a new regularization method that prevents collapse of representations and improves generalization of the model.

Abstract: Fine-tuning contextualized representations learned by pre-trained language models remains a prevalent practice in NLP. However, fine-tuning can lead to representation degradation (also known as representation collapse), which may result in instability, sub-optimal performance, and weak generalization. In this paper, we propose Representation Projection Invariance (REPINA), a novel regularization method to maintain the information content of representation and reduce representation collapse during fine-tuning by discouraging undesirable changes in the representations. We study the empirical behavior of the proposed regularization in comparison to 5 comparable baselines across 13 language understanding tasks (GLUE benchmark and six additional datasets). When evaluating in-domain performance, REPINA consistently outperforms other baselines on most tasks (10 out of 13). Additionally, REPINA improves out-of-distribution performance. We also demonstrate its effectiveness in few-shot settings and robustness to label perturbation. As a by-product, we extend previous studies of representation collapse and propose several metrics to quantify it. Our empirical findings show that our approach is significantly more effective at mitigating representation collapse.

Representation Projection Invariance
Mitigates Representation Collapse
Anastasia Razdaibiedina Ashish Khetan Zohar Karnin
Daniel Khashabi Vishaal Kapoor* Vivek Madan
@ University of Toronto Johns Hopkins University Amazon Al
anastasia raa.eiiaiaaaaaaaiiianadaaaaaaa uttoooaataaa..ooooooo.ooooooooo. www.onto.ooooomom ca danielk@jhu edu
fvivmadan, zkarnin, khetan, Aishaalk)@amazonn com
Abstract
Classification
Fine-tuning contextualized representations learned by pre-trained language models re- mains prevalent practice in NLP. However, fine-tuning can lead to representation degrada- tion (also known as representation collapse), which may result in instability, sub-optimal per- formance, and weak generalization.
Head:
Pre-treined Model Tunable
Pro tnn-traindd Model Frozen
Input Text:
Model tuning
Minimal tuning
REPINA (ours)
In this paper, we propose Representation Pro- jection Invariance (REPINA), a novel regular- ization method maintain information con- tent of representation and reduce representa- tion collapse during fine-tuning by discourag- ing undesirable changes in the representations. We study the empirical behavior of the pro- compa-
Porformance:
X
Mitigates
rep. collapse:
Figure Fine- tuning the whole architecture (left) gen- erally leads good performance though it distorts the pre-trained representations. Minimal tuning (of classi- fication head, for example; middle) mitigates the rep- resentation collapse but limits the model performance. Our proposal REPINA (right) leads to good performance while mitigating the representation collapse.
posed regularization in comparison rable baselines across 13 language understand- ing tasks (GLUE benchmark and six additional datasets). When evaluating in-domain perfor- mance, REPINA consistently outperforms other baselines on most tasks (10 out 13). Addi- tionally, REPINA improves out-of-distribution performance. We also demonstrate its effec- tiveness in few-shot settings and robustness to label perturbation. As by-product, we ex- tend previous studies of representation collapse and propose several metrics quantify it. Our empirical findings show that our approach is sigmificantly more effective at mitigating repre- sentation collapse.!
their generalizability to other domains, styles, tasks. An alternative approach to full model tuning to fine-tune only several top layers, while keep- ing the rest of the model frozen (e.g., we could train solely a classification head, Fig 1; middle). This practice of freezing all/most of the model parame- ters can prevent unwanted changes to pre-trained representations, but can also limit fine-tuning and negatively affect performance (Lee al., 20190 90; Kumar et al., 2021). This study aims determinc if it is possible to fine-tune the entire model without compromising representation quality.
1 Introduction
Fine-tuning pre-trained language models has been shown to achieve remarkable performance on va- riety of natural language processing (NLP) tasks (Kenton and Toutanova, 2019; Brown et al., 2020; Zhang al., 2022). standard fine-tuning strategy involves adapting the pre-trained model super- vised downstream task (Fig 1; left). Such a proce- dure can result representation collapse (Agha- janyan et al., 2021; Zhou and Srikumar, 2022), dis- torting the pre-trained representations that limits
We introduce Representation Projection Invari- ance (REPINA), a regularization objective that pre- vents undesirable changes in the representations (Fig 2a). Our regularization applies an invariance loss tunable projection of the representation. This regularization allows the underlying represen- tation to change mildly (e.g., shift and scaling) while not losing its expressivity (Fig 2b). Our regu- larization objective provides a knob that controls the amount of loss-free transformations allowed
'Cdde is available at https//gathhh.comm/aaaa//EEEE

---

Pre-trained Representations
Finetuned Representations
ing the fine-tuning process. A straightforward im- plementation would anchor representations during fine-tuning to their pre-trained values. That is, the final loss L would combine the standard fine-tuning objective and regularizer of the deviation in repre- sentations:
- L+
-
fpre((x))) I (z)
(1)
JEI
b)
where L is the downstream task loss (e.g., cross entropy for classification tasks), I are the input samples of the task, fpre and ffin are the repre- sentation functions defined by the pre-trained and fine-tuned networks. Optimizing full model param- eters under this modified objective would prevent representation degradation. However, this formula- tion of the loss function could be very restrictive.
Figure 2: Example shows representations collapsing into single dimension and losing useful information after fine-tuning. Example b) shows changes rep- resentations that preserve their expressive power (e.g.. coordinate shift, rotation, scaling, etc.).
Various transformations of a representation maintain its expressivity (such as linear shift; Fig 2b). While such transformations do not change the information content of representation, they incur high regularization loss based on equa- tion 1. To address this issue and allow flexibility in representations while preserving their expres- sive capacity, we propose representation projection invariance regularization (REPINA):
during fine-tuning.
We compare our method against several estab- lished regularization approaches which explicitly implicitly address the issue of representation degradation (Section 5.1). We show that ap- proach consistently outperforms major fine-tuning methods across 13 tasks (Fig 3; left) and improves generalizability on out-of-distributiin (OOD) data (Section 5.4). Our approach particularly effec- tive in scenarios where data is limited (such as with only 250, 500, or 1000 examples), as the model is more likely to overfit and memorize the train- ing data in these cases (Section 5.6). Furthermore, we thoroughly investigate fine-tuning under label perturbation (from 5% to 30% label noise) and observe that our approach robust to incorrect labels, exceeding the performance of standard fine- tuming procedure and common baseline methods (Section 5.5).
R min Σ [Ipre()) -0f==((((mm))))) DE#
L= L XR.
(2)
Here & is class of dimension preserving func- tions chosen before the fine-tuning process and defines the strength of regularization. The intu- ition behind the regularization objective is in- centivize the representations to be invariant un- der some projection & E 4; pre-trained repre- sentations can be constructed from the fine-tuned representations by applying a function & E d. For instance, if + is the set of linear functions {o 3., b: <(2) Wz + b} then we bias fine- tuned representations be linear transformations of the fine-tuned representations. Thus, regulariza- tion loss in case of Fig 2a would be zero since there exists a linear mapping from fine-tuned representa- tions to the pre-trained representations. However, regularization loss for Fig 2a would be high as there does not exist such linear mapping from fine-tuned to the pre-trained representations.
Finally, we quantify how much different meth- ods mitigate the degradation of representations (Section 6). We use previously explored probing experiments (Aghajanyan al., 2021), and pro- pose a new of metrics that quantify representa- tion collapse objectively, without requiring extra datasets/training. We observe that REPINA shows the strongest resistance to representation degrada- tion among all methods.
2 REPINA: Representation Projection Invariance
Choice of the class of functions Φ: defines the strength of the regularizer. For instance, singleton containing an identity function the strongest
Our method avoids representation collapse by pre- venting undesirable changes in representations dur-

---

GLUE
Non- GLO-
Probing Experiments
Repr. Diversity
80
600
50
500
45
GM Metric
78
85
400
300
76
200
STD+
REPINAJ
RePINAMLP
STD++
REPINAJ
REPINAMLE
STD++
REPINA
REPINAMLE
STD++
REPINA
REPINAMLE
Figure 3: REPINA improves finetuning performance and mitigates representation collapse. STD++: Improved variant of standard finetuning. REPINAJ and REPINAMLP are our methods. GLUE Non-GLUE: Average test set performance across seven GL.U and six -LUE tasks. Probing Experiments: The measure of representation collapse introduced by Aghajanyan et al. (2020) (higher is better). Representation Diversity: Mathematical measure the information content of representations (we report GM-5 score, see Section 6; higher better).
regularizer which keeps fine-tuned representations close to the pre-trained representations (equiva- lent to equation 1). Conversely, for a rich Φ e.g.. deep and wide neural networks, & can be chosen to reconstruct the lost information in representa- tion even if there is a severe degradation. Thus, it provides weak regularization. Choice of + then depends on how prone fine-tuning to over- fitting and how strong of a regularization method is needed. For instance, few-shot setting may re- quire the strongest regularization and larger train- ing datasets may require milder regularization.
discussion on the connection and the formal proof of equivalence in Appendix B.
Related Work
Mitigating Representation Collapse: Agha- janyan et al. (2020) study representation collapse and propose two methods, R3F and R4F, to address it. R3F induces bias towards a solution with a lo- cally smooth prediction function, and R4F extends R3F by adding Lipschitzness constraint on the top classification layer. Some of the other methods which implicitly target representation collapse are FreeLB and SMART. FreeLB uses adversarial train- ing to improve fine-tuning (Zhu et al., 2020) and SMART is trust region- method that avoids aggressive updates during fine-tuning (Jiang et al., 2020). R3F (Aghajanyan et al., 2020) has been shown to outperform both of these methods. Thus, only include R3F in our set of basclines.
In this paper, we experiment with & containing identity function (REPINA)) and shallow multi- layer perceptrons (REPINAMLP
Choosing the right representation: Normally, sentence-level representations are obtained from the final encoder blocks of a transformer model. However, it may be more beneficial to repre- sentations from the lower layers. In fact, Zhang al. (2020) show that re-initializing weights of the top layers encoder improves fine-tuning perfor- mance, suggesting that representation consistency may not be desirable for the top layers. Thus, we consider a variant of regularization that uses repre- sentations from the intermediate layers of encoder. Explaining representation invariance regular- ization as implicitly learning multiple tasks: Consider the overfitting in Fig 2a again. can be prevented by fine-tuning the representations for multiple tasks simultaneously instead of single task. This multi-tasking is well-known way to prevent overfitting. It not only prevents overfitting of representations but can also improves generaliza- tion performance for all of the tasks. We show that REPINA's regularization objective (equation 2) equivalent to fine-tuning on multiple hypothetical tasks. Due to space constraints, we defer further
A method that specifically targets representa- tions during fine-tuning is Supervised Contrastive Learning (SCL). SCL. induces representations of examples with the same label to be close to each other and far from the examples of other classes (Gunel al., 2020). A major disadvantage of SCL a requirement for large mini-batch size and, hence, heavy memory consumption. We imple- ment a memory-efficient SCL version but exclude the original implementation from the baselines due to computational cost (see Appendix H). Another method which can be potentially useful for mit- igating representation collapse and form part of our baseline Data augmentation. It be done via back translation, synonymn replacement, ran- dom deletion, and synthetic noise and known improve generalization performance (Feng et al., 2021; Wei and Zou, 2019).

---

Catastrophie Forgetting (Kirkpatrick al., 2017) is a phenomenon closely related to representation collapse. In a sequential training setting, it refers to forgetting information learnt from previous tasks while being trained on a new task. our con- text, this means forgetting the -reetrainig language modeling task while training for the fine-tuning task. In contrast to catastrophic forgetting, we mea- sure representation collapse as the loss in expres- sive power of the representations irrespective of the performance on pre-training task. A method known to alleviate catastrophic forgetting is Weight Consolidation (Chen et al., 2020; Kirkpatrick et al., 2017). It regularizes the fine-tuning process by encouraging fine-tuned weights to be close to the pre-trained weights. In contrast to weight consoli- dation, REPINA does not put direct constraints weight updates, but tries control the structural changes in representations.
(MLP). We the depth of MLP to 2, keeping the width equal the representation dimension. By varying the depth from to 5, we observe that for smaller training datasets, lower depth performs better. Training with large datasets robust to the depth choice (see Appendix D).
4.2 Baselines
We use a diverse range of baselines for our study: STD+ an improved variant of the standard fine- tuning scheme that includes the use of bias correc- tion in AdamW, following the works of (Zhang et al., 2020; Mosbach et al., 2020) which shows that bias correction is a major cause of instability in language model fine-tuning.
Weight Consolidation (Kirkpatrick et al., 2017: Chen et al., 2020) is an approach that encour- ages agreement between pre-trained fere and fine- tuned o models weights via a regularization term Σ, 2.11///m - -,-e added to the loss function.
Due to our limited space, we discuss further de- tails on related works in Appendix A.
R3F (Aghajanyan et al.. 2020) is local smooth- ness regularization that prevents aggressive model updates by restricting divergence of outputs upon input perturbation. For model f(-) and input to- ken embeddings x, R3F adds regularization term KLs (fr)) ff + +)) the loss function, where KLg is the symmetric Kullback-Leible divergence and noise 2 sampled from normal distribution. ReInit (Zhang et al., 2020) improves fine-tuning performance by re-initializing the top-k layers of the encoder (closer the output) with gaussian random samples from (O, 0.02²). Following the original study, we perform hyperparameter search for k = 2, or 6.
4 Experimental Set Up
4.1 Our Methods: REPINA] & REPINAMLP
Recall our methods REPINAJ and REPINAMLP in- troduced in Section 2. For REPINA, we observe that regularizing intermediate layer representations (5th, 10th, 20th from input) perform better than reg- ularizing the top layer (near the output, before the classifier) representation. Thus, the regularization objective for REPINAJ is:
R(Φ - (1}) = > ff IEI
Data Augmentation (DA) generates augmented samples by adding noise to the training data (keep- ing the label intact) (DeVries and Taylor, 2017). In our implementation, we add gaussian noise e (0, 8) to the token embeddings where 1e-5. 5. Table 2 show the regularization coefficients used for each method.
where ffres Jfmm are 8-th representations from the f-th layer (from input) of the model. Choice of & is a hyper-parameter with possible values of 5, 10 and 20. Layer 5 is most effective for small training datasets and layer 20 is most effective for large training datasets (see Appendix E).
Due to computational limitations, we experi- ment with only top layer representations for RF- PINAMLP Thus, the regularization loss for RE- PINAMLP. , R(Φ = MLP) is:
4.3 Datasets
We evaluate methods on GLUE benchmark (Wang et al., 2018) and six additional non-GLUE datasets (Table 3). These include: biomedical relation ex- traction on CHEMPROT (Kringelum et al., 2016), sentiment classification on YELP (Zhang al., 2015a) and IMDB (Maas al., 2011), citation intent classification on SCICITE (Cohan et al., 2019), language inference on SCITAIL (Khot
min Ellfr((x) 1
where Spres fin are the representations from the top layer of the model (before the classifier) and è are the parameters of a multi-layer perceptron

---

Method Tass +
RTE
MNLI
SST2
MRPC
QNLI
QQP
CoLA
Yelp
Chem
IMDB
AGnews
SciTail
SciCite
STD++
70.8
65.6
92.1
86.8
87.2
76.7
59.70
95.3
82.6
93.2
91.7
71.6
81.9
DA
73.6
65.5
92.0
90.7
87.4
76.4
63.4
95.6
82.9
93.2
91.8
93.7
82.1
WC
72.2
66.7
92.7
88.6
87.2
76.2
61.5
95.9
83,9
93.4
91.9
94.0
82.2
ReInit
70,9
65.1
92.0
91.0
87.3
77.2
61.2
95.4
82.5
92.7
91.7
93.4
82.4
R3F
70.4
65.0
92.1
89.9
87.0
74.9
62.0
95.5
82.9
93.1
91.7
86.5
82.0
REPINAL
71.4
65.7
92.9
91.5
87.5
79.0
62.3
95.8
83.5
94.0
92.1
93.7
82.7
REPINAMLP 74.4
65.2
93.2
91.1
87.6
79.3
62.5
96.0
83.7
93.9
91.9
94.8
83.2
Table I: Performance for our methods (REPINAMML) and baselines on 7 GLUE and non-GLUE datasets. Average gain of 2.1 over STD++ for GLUE datasets and 4,5 over non- non-GLUE datasets. REPINAbeats all baseline methods in 10/13 cases. For QQP, MNLI, QNI, AGNEWS, IMDB, YELP and SCITAIL, we only used 10K training datapoints.
Method
Regularization coefficient
4.4 Fine-tuning Settings
REPINAJ
0.01, 0.05, 0.1, 0.5
Due to the large scale of the experiments and in order to have meaningful comparison with var- ious approaches, we consistently use BERT-large model for implementing both our proposed algo- rithm and the baselines. Existing works such (Zhang et al., 2020; Mosbach et al., 2020) also use similar experimental setups. Additionally, to verify the generality of our findings to other models, we performed limited experiments on Ba-aaaee where we observe similar performance gain.
REPINAMLP
0.01, 0.05, 0.1, 0.5
DA
0.05, 0.1, 0.2, 0,4, 0.8
R3F
0.1, 0.5, 5,5
WC
0.01, 0.05, 0.5
Table 2: Regularization coefficient for different meth- ods.
et al.. 2018) and article topic classification on AG- NEWS (Zhang et al., 201 For each task, we use their corresponding adopted performance metric
We fine-tune all models for 5 epochs (unless oth- erwise specified) at a learning rate of 2e-5, and report performance with 5 different seeds. Due to resource constraints and in contrast to prior works (Kenton and Toutanova, 2019; Aghajanyan et al., 2020; Chen et al., 2020), we do not search for optimal learning rate for each method-task combi- nation. To verify the impact of this choice, we per- form limited experiments selecting the best learn- ing rate, dropout and number of epochs for each method and a subset of tasks (Appendix F). We observe similar gains as reported in the main pa- per. For each method, we select optimal hyperpa- rameters by performing evaluation on the unused fraction of the training set (see Appendix C).
On these 13 datasets, we conduct a variety of ex- periments with many and few supervision instances, To keep the cost of fine-tuning computations ex- tremely large datasets (such as MNLI and QQP), we limited their training sets to 10, 000 data points, and marked with suffix "-10K" henceforth. For datasets with no available test set labels, we use their development set to report the performance. We use a subset of original train data split (size equal to validation set) which is not used for train- ing for hyper- paremmeer selection.
Task
Train
Dev
Metrie
COLA
8551
1043
2
MCC
Since standard fine-tuning is susceptible failed runs that substantially lower the resulting perfor- mance (Mosbach al., 2020; Razdaibiedina and Brechalov, 2023), we filter out failed runs and re- port average performance over 5 successful runs. We define run as failed if its performance is close to or lower than the majority classifier (i.e. dummy model that always predicts the label of the majority class in the dataset) (Dodge et al., 2020). We define threshold close to the performance of the majority classifier as per metric in Table 3. A fine-tuning run is "failed" if its performance on unused part of the training dataset is below the threshold. See Section C.2 for the exact thresholds.
RTE
2490
277
2
Accuracy
SST
67349
872
2
Accuracy
MNLI-1OK
10000
9815
3
MCC
MRPC
3668
408
2
F1
QQP-10k
10000
40430
2
Fl
QNLI-1OK
10000
5463
2
Accuracy
CHEMPROT
4169
2427
13
Micro F11
SCICITE
7320
3
Macro Fl
SCITAIL--IO
10000
1304
2
Accuracy
AGNEWS- 10k
10000
5000
4
Macro F1
YELP-1OK
10000
10000
2
Accuracy
IMDB-1OK
10000
5000
2
Macro Fl
Table The datasets used this study, their size, num- ber of classes (C) and the corresponding evaluation met- rics. MCC denotes Matthews correlation coefficient.

---

5 Results: Generalization Performance
5.3 Comparison with parameter-efficient tuning methods
In this section, we present experimental results with the baselines introduced in the earlier section.
The main focus of our study is mitigating repre- sentation collapse in full model tuning setting, which is a common way to tune LLMs since it allows to achieve the best downstream task perfor-
5.1 Full dataset Generalization performance Table shows that REPINA models outperform the baselines consistently across a variety of tasks: our method outperforms other ones on 10/13 tasks. Both REPINA and REPINAMLP outperform baseline methods in terms of mean performance, with improvements in the mean performance over the corrected fine-tuning strategy STD++ by 1.7 and 2.0 points, respectively, for GLUE benchmark, and 4.3 and 4.5 points for non-GLUE benchmark.
mance, at the cost of forgetting and representation collapse. However, this goal can also be achieved with parameter-ecficient tuning (PEFT) methods. which balance good downstream performance with minimal parameter changes.
As we have discussed in the introduction, if fine-tune only several top layers of the model, while keep- ing the rest of the parameter frozen (e.g., train solely classification head, Fig 1; mid- dle), representation collapse can avoided. How- ever, this would not lead to optimal downstream task performance. As middle ground approach, could train just larger fraction of model pa- rameters (yet not the full model) with parameter- efficient approaches. Common PEFT methods in- clude LoRA (Hu al., 2021), prefix tuning (Li and Liang. 2021), prompt tuning (Lester et al., 2021),
5.2 Analyses Fine-tuning Stability
Similar to the prior literature (Dodge et al., 2020; Mosbach et al., 2020; Wang et al., 2018), we ob- serve that the standard fine-tuning procedure is prone to instability and sub-optimal convergence, leading to failed runs. Recall that we formally de- fine fine-tuning run failed run if the resulting performance is close the majority classifier. In the previous section, reported the mean performance of only successful runs (complement failed runs). Figure shows the fraction of runs that were successful for each method. We note that REPINAJ has the least number of failed runs (maxi- mum number of successful runs). Moreover, if we do not filter out failed runs, our methods perform even better than all the baseline methods. REPINAI achieves average 2.6 percentage point improve- ment over the next best baseline method (WC). Thus, we conclude that our methods demonstrate higher stability and less fraction of failed runs than other approaches. (additional experiments in Table 21 in Appendix K.)
adapters (Houlsby et al., 2019), and variations those (Rücklé et al., 2020; Khashabi et al., 2021; Lester et al., 2022; Razdaibiedina et al., 2023a,b).
We provide results for prompt tuming, prefix tun- ing, LoRA and REPINA-MLP on 4 GLUE tasks in Table 4. As we can see, REPINA-MLP consis- tently outperforms PEFT methods.
SST-2 MRPC CoLA
RTE
Prompt Tuning
86.4
76.9
59.9
52.5
Prefix Tuning
90.9
91.0
57.6
73.9
LoRA
91.5
90.0
60.4
71.5
REPINA-MLP
93.2
91.1
62.5
74.4
Fraction of successful runs
Table 4: Comparison with PEFT methods.
0.9
0.8
5.4 Out-of-distributtion robustness
0.7
We perform experiments on four pairs of datasets to study OOD generalization following Hendrycks et al. (2020) protocol (see Table 5). The chosen pairs of datascts induce distribution shift, which allows to measure OOD robustness. Due limited resources and different sizes of tasks, limited the training set 200 examples per class (fixed across all runs). Overall, REPINA shows steady improvement in OOD performance cases.
0.6
RPM
RP1
Figure 4: Fraction of successful runs across all tasks. Run is defined successful if its test performance is higher than the performance of a majority classifier. Our proposed regularization (RP1 /RPM) increases the fraction of successful runs, hence, leading to more stable fine-tuning behavior.

---

Task
IID
OOD
A method's rank for task corresponds to the posi- tion of the method in list of all methods sorted by performance on that dataset. The minimal and best possible rank is 1. The average rank of a method is obtained by averaging ranks across all tasks.
Train + Eval
STD++
RP1
STD++
RPI
Imdh + -ST22 92.1
92.1
87.810.9
88.9
SST2 Imdb 91.1+0.1
92.1-o.1
87.1
87.8 0.1
Yelp Amzn 60.7.0.2
60.8
28.940.2
29.3
Amzn - Yelp
39.4+1.9
41.5-1.4 49.741.9
5..8-14
We observe in Table 7 that REPINAT is the most effective method in the few-shot setting mea- sured in terms of the average rank. See Ap- pendix J for a detailed analysis.
Table 5: REPINA improves OOD performance. RP1
REPINA Train: training task (IID), Eval: evaluation- only task (OOD). Data limited to 200 samples/class.
5.5 Robustness to Label Perturbation
Real-world data can often contain mislabeled sam- ples, which can hinder the training process. Hence, robustness to label noise is desirable quality of the fine-tuning approaches. Here, we study the performance of the fine-tuning methods under la- bel perturbation. We introduce label noise as fol- lows: let C = {1,...;<} .c be a class of labels and X = {(2, 3)} be the true dataset for the fine- tuning task. Our fine-tuning method has access to noisy dataset X' = {(2, y')} where y' = with probability - p and sampled uniformly from {1 {I......c} } with probability p.
samples
STD++
DA
WC
Relnit RP|
R3F
RPM
250
5,62
4.92
4.62
3.00
4.04
2.50
3.31
500
6.08
4.38
3.69
3.31
4.85
2.77
2.92
1000
5.69
4.00
3.62
3.54
4.62
2.69
3.85
Table 7: Average rank of different methods for shw-shot learning. RP1 : REPINAI RPM REPPIAAMP
Overall, we find that REPINA MLP yields perfor- mance gain on large-scale datasets, whereas RE- PINAI is effective for few-sample fine-tuning (since newly introduced parameters in REPINAMLP are undertrained when the training data is limited). For wall-time analysis, see Appendix O. For experi- ments on hyper-paramett optimization over learn- ing rate, batch size and other papeer-arammters see Appendix F.
REPINAI and REPINAMLP show the high- est resistance to label perturbation, retaining closest to the original performance upon introduc- ing 5-10% noise to labels (Table 6). The second most resistant approach, WC, is also close to our method conceptually, as it discourages the fine- tuned weights to deviate from pre- pre-trained weights.
Degradation of Representations:
Analytical Perspective
Here we quantify the representation collapse.
Noise + STD++
DA
WC
ReInit
R3F
RP1
RPM
6.1 Probing Representation Collapse
0%
64.7
78.5
81.4
79.9
72.9
84.0
83.0
We follow the setting of Aghajanyan et al. (2020) for studying representation collapse with probing experiments as follows: (i) fine-tune model on a downstream task A, (ii) freeze the encoding layers and train the top linear layer for a different task B. Low performance in the second step implies representation collapse in the first step. To assess robustness of the proposed approach to represen- tation collapse, we perform series of probing ex- periments. In our experiments, we use four GLUE and four non-GLUE datasets in the first step and all datasets in the second step except the one used in the first step (Table 8).
5%
58.3
68.2
75.3
72.3
57.3
81.4
78.0
10%
58.0
63.7
72.2
68.9
52.4
78.1
75.6
20%
48.4
49.1
64.1
55.2
44.3
66.2
70.2
30%
40.1
45.9
53.5
52.4
42.0
50.3
59.5
Table Mean performance over 13 datasets when train-
ing with noisy data. RP1: REPINAI RPM RE-
PINAMLP See Appendix L for detailed results.
5.6 Analyses on Few-shot Fine-tuning
To investigate our methods' robustness to small dataset sizes, we study REPINAMLP and REPINAJ performance in limited data settings (250/500/1000 training data points). We fix the same data subset across all models to avoid performance changes related to data variability.
We observe that REPINAMLP and REPINA show high resistance to representation collapse, out- performing other approaches in 6/8 cases (Table 8). For instance, fine-tuning for QNLI-10K in the first step with REPINAMLP results in a mean perfor- mance of 49.5 in the second step, whereas the next best baseline results in a mean performance of 44.5.
Since finetuning in few-shot setting is particu- larly prone instability and the performance on a single dataset can distort the mean statistics for the entire collection, we use average rank as a more stable metric to compare different methods.

---

Task
STD++
DA
WC
Relnit
R3F
RPI
RPM
V(Ae)
QNLI
37.6
37.1
44.5
37.7
36.5
41.7
49.5
V(As)
V(1))
QQP
39.8
42.6
44.5
41.2
36.3
52.4
44.1
RTE
32.0
32.0
37.2
48.9
33.3
51.9
42.0
MNLI
36.3
40.6
41.3
52.5
43.0
51.0
48.5
V(ha)
AG
41.1
42.5
42.3
43.3
41.4
43.7
47.8
V(A2)
V(As)
V(he)
V(A.)
v(As)
IMDB
45.2
44.0
43.3
42.0
44.5
47.9
48.4
A3
A2
SCIT
39.0
50.3
46.2
44.6
34.0
47.8
48.8
SCIC
43.9
44.7
46.3
41.4
39.1
48.3
48.1
Top-20 eigenvalues distribution
10a
Aver.
39.4
41.7
43.2
44.0
38.5
48.1
47.1
STD++ REPINA
Table 8: Results of probing experiments to measure representation collapse (higher score is better). Model is fine-tuned for task A with different methods, then new linear head is trained for the remaining 12 tasks and the mean performance is reported. Aver. is aver- age over different choices of task A. RPj is REPINA RPM is REPINAMLP AG: AGNEWS- 10k, SCIT: SCITAIL-IOK, SCIC: SCICITE-IOK QNLI: QNLI-10K, QQP:QQP-IOK, MNLI: MNLI-IOK.
valua
10
10a
1 2
4
7
9 10
11
12
13
14
15
16
17
18
19
20
Elgenvalue number
Figure 5: Top: A, and V(A.) correspond to ith eigen- value and its associated eigenvector after eigendecompo- sition of Gram matrix. Data from the left distribution is well spread out and spans all three dimensions, with all of its eigenvalues being similar. The right distribution shows all of the data collapsed along eigenvector, hence one of the cigenvalues significantly exceeds two others. Bottom:
Note that auxiliary tasks considered here used only to evaluate the degradation of represen- tations. They are not available during finetuning. During fine-tuning stage, only one task dataset available. Thus, we do not compare our methods to the rehearsal-basee learning methods.
comparison of top-20 eigenvalues of STD++ and REPINAJ after fine-tuning QQP with 250 points. Less skewed distribution of eigenvalues for REPINAJ compared STD++ indicates more spread out distribution of fine-tuned representations with RE- PINAj compared to STD++.
6.2 Diversity of Fine-tuned Representations
Probing experiments rely on the availability of ex- tra fine-tuning tasks and, thus, are limited in the amount of information they can assess, requiring additional fine-tuning rounds. Here, we propose metries that can reliably quantify the power of fine- tuned representations by capturing their geometric diversity. The intuition behind our metrics is the following: if all representations lie in a small di- mensional space such as a straight line or a sin- gle point, then they contain little information and are not expressive. But if representations are well spread out and span the entire representation space, then they possess high information capacity).
one cigenvalue being substantially higher than the others. Overall, more uniform distribution of the eigenvalues corresponds to a better representation matrix diversity. In the bottom bar-plot we show distribution of the top-20 eigenvalues of the fine- tuned representations with REPINAJ and STD++ after training on QQP dataset with 250 points (Fig- ure 5). REPINAJ preserved a closer to uniform eigenvalue distribution, while STD++ results in representations with much higher first eigenvalue, indicating representation collapse. Thus, REPINA yields better representation matrix diversity and less representation collapse than STD++
We illustrate representation collapse metrics from the geometric perspective in Figure 5. The top three plots show three different distributions of data points (representations). The left distribution spans all three dimensions, indicating the highest degree of data diversity. Since data points equally lie in all dimensions, all three eigenvectors (V (V())'s) will be of equal importance and all three cigenvalues (Az's) will be approximately equal. In contrast, the central distribution spans two axes, leading smaller third eigenvalue that corresponds to the "re- dundant" dimension. Right distribution has all the data points collapsed along one axis, resulting in
Next, we formalize this intuition by defining a representation diversity metric based on the geo- metric diversity of sentence-level representations.
Diversity Metries:
We compute the gram ma- trix G for the representations where Gtj (xs), ffin(x,))- f From we obtain eigen- values A1 2 ... 2 Aa- To measure diversity representations, we use geometric mean (GM) and

---

harmonic mean (HM) of the eigenvalues:
tasks, including few-shot and label noise settings, and improve generalization performance. We also study the degradation of representations during fine-tuning, representation collapse, and propose new metrics to quantify it. Our methods reduce rep- resentation collapse and improve OOD robustness.
GM (I An) 1/d
Determinamt
HM E X
Trace (G (G-1)-
These metrics attain high value if the represen- tations are well spread out and are zero or close zero if all/most of the representations lie in smaller dimension subspace. In contrast to arith- metic mean, geometric and harmonic mean are not as sensitive to outliers. We observe that these met- rics turn out to be always zero representations typically lie in 20-dimensional space. Hence, we chose top-k A values for k 5, 10, 20 where GM and HM are bounded away from 0.
Limitations
We conduct extensive experiments in our paper and show that the proposed approaches lead to sig- nificant gains. However, we did not exhaust all avenues of investigation due to limited resources. Firstly, we could experiment with different choices of & other than REPINA (Ø is identity) and RE- PINAMLP (ø is MLP). Other choices of á may include deeper networks or transformer-based mod- els, which could potentially improve performance even further. Secondly, we investigated how rep- resentations from intermediate layers affect RE- PINAJ performance, and observe major improve- ments with top layer representations. Similar ex- periments for REPINAMIP may also yield further gain. Also, in REPINAI we could experiment with more choices of the representations layer (we tried 5th, 10th, 20th layer). Since lower layer repre- sentations are more computationally efficient to regularize (do not require full forward pass through the model), another interesting direction is finding trade-off between computational efficiency of the regularizer and performance gains.
(HE
(nI
1 HM-K-
GM-k
We compare REPINAJ and REPINAMLP to the
Metric 4 STD+
DA
WC
ReInit
R3F
RP
RPM
GM-5
396
484
425
397
463
GM-10
118
118
90
91
GM-20
14
13
13
HM-5
198
253
242
184
207
290
217
HM-10
47
37
32
HM-20
5
3
3
3
Table Diversity of fine-tuned representations. Mean value across all the 13 tasks is presented. RP1 is RE-
PINAI RPM is REPINAMLP- REPINAJ yields fine-
tuned representations with maximum representation trix diversity.
This study primarily focused on medium- sizid models due to computational constraints. It is the outcome of extensive experimentation, which would have been impractical with limited computa- tional resources. Although we have experimented with masked language models, believe the find- ings apply to other architectures that follow similar principles. We anticipate that future research will provide more insight into these issues.
existing baselines using GM-k and HM-K with h = 5, 10, 20 (Table 9). Low GM-k and HM-k indi- cates representation collapse, when fine-tuned rep- resentations lie a low-dimensional space. High GM-k and HM-k indicates that representations are well spread out and span a higher dimensional space. Table 9 shows that REPINA results in the most diverse representations among all the baseline methods and incurs least representa- tion collapse(see Appendix for detailed results).
Ethical Considerations
REPINA aims to improve performance and retain the quality of representations during fine-tuning. Practically, our method can help in suppressing po- tential biases of language models after fine-tuning datasets that include biased content. REPINA can achieve this by reducing collapse of represen- tations and preserving their pre-trained knowledge.
7 Conclusion
In this paper, we propose a novel representation invariance regularizer targeted at avoiding repre- sentation degradation during finetuning. It has knob that can control strength of regularization. We experiment with two choices of this knob, RE- PINAi and REPINAMLP and show that they both achieve significant performance gain across 13
All our experiments are based on publicly available datasets and, hence, there is no immediate concern about harmful content.

---

Acknowledgments
We thank the anonymous reviewers for their con- structive feedback. DK is supported by generous gifts from Johns Hopkins University, Allen Insti- tute for Al and Amazon. AR is supported by Vector Institute for Artificial Intelligence and the Univer- sity of Toronto.

---

References
Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan. Pascal Germain, Hugo Larochelle, Francois Lavi- olette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversen training of neural networks. The journal of machine learning research, 17(1):0096- 2030.
Steven Abney. 2007. Semisupervised learning for com- putational linguisties. CRC Press.
Armen Aghajanyan, Sonal Gupta, and Luke Zettle- moyer. 2021. Intrinsic dimensionality explains the Pro-
effectiveness of language model fine-tuning. ceedings of the 59th Annual Meeting of the Asso- ciation for Computational Linguisties and the 11th Interaatoonal Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 7319- 7328.
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural net- works. In Proceedings of the thirteemth international conference on artificial intelligence and statistics, pages 256. JMLR Workshop and Conference Proceedings.
Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, and Sonal Gupta. 2020. Better fine-tuning by reducing representational collapse. In International Conference on Learning Representations.
Beliz Gunel, Jingfei Du, Alexis Conneau, and Veselin Stoyanov. 2020. Supervised contrastive learning for pre-trained language model fine-tuning. In Interna- tional Conference on Learning Representations.
Eyal Ben-David, Carmel Rabinovitz, and Roi Reichart. 2020. Perl: Pivot-based domain adaptation for pre-trained deep contextualized embedding models. Transactions of the Association for Computational Linguistics, 8:504-521.
Suchin Gururangan. Ana Marasović, Swabha Swayamdipta, Kyle Lo, 1z Beltagy, Doug Downey, and Noah A Smith. 2020. Don't stop pretraining: Adapt language models to domains and tasks. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3360-
John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspon- dence learning. In Proceedings of the 2006 con- ference on empirical methods in natural language processing, pages 120-128.
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, and Dawn Song. 2020. Pretrained transformers improve out-of-distribttion robustness. arXiv preprint arXiv:2004.06100.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019, Parameter-efficient transfer learning for nip. In In- ternational Conference on Machine Learning, pages 2790-2799, PMLR.
Sanyuan Chen, Yutai Hou, Yiming Cui, Wanxiang Che, Ting Liu, and Xiangzhan Yu. 2020. Recall and learn: Fine-tuning deep pretrained language models with less forgetting. In Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), pages 7870-7881.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adap- tation of large language models. arXiv preprint 1rXiv:216600968:
Arman Cohan, Waleed Ammar, Madeleine Van Zuylen, and Field Cady. 2019. Structural scaffolds for cita- tion intent classification in scientific publications. In NAACL.
Haoming Jiang. Pengcheng He, Weizhu Chen, Xi- aodong Liu, Jianfeng Gao, and Tuo Zhao. 2020. Smart: Robust and efficient fine-tuning for pre- trained natural language models through principled regularized optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2177-2190
Terrance De Vries and Graham W Taylor. 2017, Dataset augmentation in feature space. arXiv preprint 702.588....
Jesse Dodge, Gabriel Ilharco. Roy Schwartz, Ali Farhadi, Hannanch Hajishirzi, and Noah Smith. 2020. Fine-tuning pretrained language models: Weight ini- tializations, data orders, and early stopping. arXiv preprint arXiv:2002.00 06305.
Jacob Devlin Ming-Wes Wei Chang Kenton and Lee Kristina Toutanova, 2019. Bert: Pre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171 1-4186.
Steven Y Feng, Varun Gangal, Jason Wei. Sarath Chan- dar, Soroush Vosoughi, Teruko Mitamura, and Ed- uard Hovy. 2021. A survey of data augmentation approaches for nlp. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 968-988.
Daniel Khashabi, Shane Lyu, Sewon Min, Lianhui Qin, Kyle Richardson, Sean Welleck, Hannaneh Ha- jishirzi, Tushar Khot, Ashish Sabharwal, Sameer Singh, et al. 2021. Prompt waywardness: The curi- ous case discretized interpretation continuous prompts. arXiv preprint arXiv:2112.08344

---

Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. SciTail: A textual entailment dataset from science question answering. In AAAI.
Vivek Madan, Mohit Singh, Uthaipon Tantipongpipat. and Weijun Xie. 2019. Combinatorial algorithms for optimal design. In Conference Learning Theory, pages 2210-2220222 2258. PMLR.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Ag- et al. 2017. Over-
Amil Merchant, Elahe Rahimtoroghi. Ellie Pavlick, and Ian Tenney. 2020. What happens to bert embeddings during fine-tuning? In Proceedings of the Third BlackboxNI.P Workshop on Analyzing and Interpret- ing Neural Networks for NLP. pages 33-44.
nieszka Grabska-Barwinkke coming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114((3):352133522
Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Pro- ceedings of the ACL 2010 Conference Short Papers, pages 220-222, Uppsala, Sweden. Association for Computational Linguistics.
Simon Korblithh, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. Similarity of neural network representations revisited. In International Conference on Machine Learning, pages 3519-3529. PMLR.
Marius Mosbach, Maksym Andriushchenko, and Diet- rich Klakow, 2020. On the stability of fine-tuning bert: Misconceptions, explanations, and strong base- lines. In International Conference on Learning Rep- resentations.
Jens Kringelum, Sonny Kim Kjaerulff. Søren Brunak, Ole Lund, Tudor Oprea, and Olivier Tabourcau. 2016. Chemprot-3.0r 3.0: a global chemical biology dis- eases mapping. Database, 2016.
Sinno Jialin Pan, Xiaochuan Ni, Jian-Tao Tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-domain senti- ment classification via spectral feature alignment. In Proceedings of the 19th international conference on World wide web, pages 751-760.
Ananya Kumar, Aditi Raghunathan, Robbie Matthew Jones, Tengyu Ma, and Percy Liang. 2021. Fine- tuning can distort pretrained features and underper- form out- difribstiiuuion.. In International Conference on Learning Representations.
Jason Phang. Thibault Févry, and Samuel R Bowman. 2018. Sentence encoders on stilts: Supplementary training on intermediate labeled-data tasks. arXiv preprini 1r10vo887.
Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang- 2019a. Mixout: Effective regularization to finetune large-scale pretrained language models. In Interna- tional Conference on Learning Representations.
Anastasia Razdaibiedina and Alexander Brechalov. 2023. Miread: Simple method for learning high- quality representations from scientific documents. arXiv preprint 205.v:2305.04177.
Jaejun Lee, Raphael Tang, and Jimmy Lin. 2019b. What would elsa do? freezing layers during transformer fine-tuning, arXiv preprint arXiv: 191Xiv:991..000 1et::09010000 1.03990.
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021,
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Ma- dian Khabsa, Mike Lewis. and Amjad Almahairi. 2023a. Progressive prompts: Continual learning for language models. arXiv preprint arXiv:2301.123/1
The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.0862)
Brian Lester, Joshua Yurtsever, Siamak Shakeri, and Noah Constant. 2022. Reducing retraining by re- cycling parameter-efficient prompts. arXiv preprint S.Xiv:2208.05577.
Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Ma- dian Khabsa, Mike Lewis, Jimmy Ba, and Amjad Almahairi. 2023b. Residual prompt tuning: Improv- ing prompt tuning with residual reparameterization. arXiv preprint arXiv:2305.080 305.03937.
Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.0011 101.00190.
Subendhu Rongali, Abhyuday Jagannatha, Bhanu Pratap Singh Rawat, and Hong Yu. 2020. Contin- ual domain-tuning for pretrained language models. arXiv preprint 2xXY:2204.002288
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Lin- guistics.
Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer. Nils Reimers, and Iryna Gurevych. 2020. Adapterdrop: On the effi- ciency of adapters in transformers. arXiv preprint wxXi:2010.11913
Vivek Madan, Ashish Khetan, and Zohar Karnin. 2021. Tadpole: Task adapted pre-training via anomaly de- tection. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing (EMNLP).
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference on Learning Representations.

---

Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen. and Eiichiro Sumita. 2017. Instance weighting for neural machine translation domain adaptation. In Proceedings of the 2017 Conference Empirical Methods Natural Language Processing, pages 1482-1488.
Jason Wei and Kai Zou. 2019. Eda: Easy data augmenta- tion techniques for boosting performance on text clas- sification tasks. In Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 6382-6388.
John Wu, Yonatan Belinkov, Hassan Sajjad, Nadir Dur- rani. Fahim Dalvi, and James Glass. 2020. Similarity analysis of contextual word representation models. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, pages 4638- 4655.
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint 205.0:2005.0166.
Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Wein- berger, and Yoav Artzi. 2020. Revisiting few-sample bert fine-tuning. In International Conference on Learning Representations.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015a. Character-level convolutional networks for classi- fication. Advances in neural information processing systems, 28.
Xiang Zhang. Junbo Jake Zhao, and Yann LeCun. 2015b. Character-level convolutional networks for text clas- sification. In NIPS.
Yichu Zhou and Vivek Srikumar. 2022. A closer look how fine-tuning changes bert. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 1046-1061.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun. Tom Gold- stein, and Jingjing Liu. 2020, Freelb: Enhanced ad- versarial training for natural language understanding. In ICLR.

---

Supplementary Material
Rongali al. (2020) show that rehearsal based learning can improve performance and perform better than Weight Consolidation. However, even though our method is inspired by multi-task learn- ing and performs pseudo multi-task learning implic- itly, we do not have access to any datasct additional to the single fine-tuning task. Thus, rehearsal based learning does not apply in our setting.
A Additional Related works
Due to limited space in the main text, part of the related is below. We will reintroduce these to the main text upon having more space.
Domain shift between pre-training and finetun- ing data: Even though pretrained models achieve high performance for a large number of NLP tasks, they tend to suffer if there is significant domain shift between the pretraining data and finetuning data. Domain Adaptation bridges this gap by adapt- ing the model to the finetuning task domain. can done by doing additional pre-training on task domain data if such data is available (Gururangan et al., 2020) or algorithmically finding such data from general domain corpus if such data is not available (Madan et al., 2021).
Domain shift between finetuning train data and evaluation data: Domain Adaptation typi- cally refers to the scenario where labeled train data is available in one domain and the evalu- ation is done for data in other domain. Tech- niques for addressing domain shift include model- centric techniques, data-centric techniques and hy- brid techniques. Model-centric technique changes the model by changing the feature space, loss func- tion or the structure of the model (Blitzer et al., 2006; Pan et al., 2010; Ganin et al., 2016; Ben- David et al., 2020). Data-centeric approaches in- volve pseudo-labeling (Abney, 2007), using aux- iliary tasks (Phang et al., 2018), and data selec- tion (Moore and Lewis, 2010; Wang et al., 2017). Mixout (Lee et al., 2019a) is variant of Dropout regularization that replaces dropped neurons with the pre-trained model neurons, thereby mixing pre- trained and fine-tuned parameters.
Measures of representation: Aghajanyan et al. (2020) measures the quality of finetuned represen- tations by fitting them auxiliary tasks. CKA (Ko- mblith et al., 2019) measures correspondences be- tween representations from different network. Wu et al. (2020) study the similarity of internal repre- sentation and attention of different trained models using some new similarity measures.
Merchant et al. (2020) also studies what hap- pens during finetuning via probing classifiers and representation similarity analysis. It argues that finetuning does not necessarily incurs catastrophic forgetting. It analyze the effect for finetuning dif- ferent tasks on the changes in representation.

---

B Theoretical Motivation and Connection to Pseudo Meta-learning
If g comes from distribution D over F, then our representation loss Eg [4g]- Here, we con- sider F to be the set of regression tasks?, which are characterized by vectors in {u E IRa) and tasks to be sampled from a standard Gaussian distribution with mean 0 and umit variance. We consider loss to be the natural l loss function.
Idea: We view the pre-trained model as a multi- task learner with an infinite number of pseudo tasks Ti.T2..... That is, for each there exists a lin- ear layer that fits pre-trained representations to a pseudo task T. Our aim is to fine-tune the represen- tations on specific downstream task B while pre- serving their ability to perform well on Ti, T2....... .... namely, there must exist linear model on the fine- tuned representations for each pseudo task T, The linear classification head for Ti does not have to be the same for the pre-trained and fine-tuned repre- sentations, but their output should be close.
N min vERd j=1
C = -
(u Tu=22p upre o fon
The inner minimization has closed form solu- tion (Z),n in Zfrn) ZpreZ and the resulting expectation can be reduced get:
Zpre
L <= fon) Zjen In Zpre
consistency loss on pseudo-task
consistency loss on pseudo-task
loss
where Zpre E BR Nxd matrix has j-th row aipre and Jim E Nxd matrix has j-th row 2/i 11-I12 for a vector denote the f--norm and for a matrix denote the Frobenius norm. A denote the pscudo-inverse of a symmetric matrix A. The derivation of and reduction of expectation can be found in Theorem in Appendix B. Loss function L. is not easily de- composed into mini-batches, making it challenging to optimize directly. We find an equivalent opti- mization problem whose objective is decomposable into mini-batches and whose optimum is equivalent to the representation loss L:
current
head
head M
head
head M
fine-tuning encoder
pre-trained model
encoder
model
Figure 6: Intuitive explanation of the proposed ap- proaches from the multi-task learning perspective. The total loss consists of the cross-entropy loss for the fine- tuning task and the consistency losses for the pseudo- tasks. The pre-trained model is non-trainable (frozen).
Let the training samples be F1,..., ZN for the fine-tuning task and zirre: =d E Rd be the rep- resentations (output of the encoder layer) for the pre-trained model and the model being fine-tuned. Let F be family of functions operating repre- sentations such that each function signifies a task. objective as follows:
2
È WCRax
min
Σ
Sre
W fin
We minimize E for W along with the fine- tuning objective. Derivation of the above equiva- lence can be found in Theorem 2 in Appendix B.I. We can interpret the above loss as follows: There exists a linear function (ow : I x= Wx) which operated on fine-tuned representations results in pre pre-trained representations. We can generalize it to include class of functions 4:
T.. We formalize For any function g F on pre-trained representa- tions, there must exists a corresponding fine-tuned representations giving the same output; Vg E F,3h E F s.t. g oo Zpre
F on
a 2fin During fine-tuning, we do not expect an exact agreement and allow representations to lose some expressive power. Hence, relax the constraint and consider the representation loss e
N
= min Σ - 2b ((
(3)
N
For E FFF min bEF
loss(g( (2fre): hi h((://)))
This corresponds to the aggregate loss for pseudo- tasks T, ss instead of using a linear head for
Ti's were actual pre-training tasks and the data was is available, we would compute the loss on the input of T. In absence of that, we approximate it by loss on (unlabeled) input of the given fine-tuning task.
IIf we assume the tasks to be classification, then the linear layer is followed by softmux layer. However, for simplicity we assume the pre-training to be done on regression tasks as this yields closed from expressions and yields good results.

---

pseudo tasks on fine-tuned representations, we use a function E & followed by u,- Thus, & de- fines the strength of the regularizer. A singleton & containing an identity function enforces the use the same linear head u; for task 7; on both pre- trained and fine-tuned representations. This results in the strongest regularizer which keeps fine-tuned representations close to the pre-trained represen- tations. On the other hand, if Φ is set of very deep neural networks, then we allow deep neural network (+) fit fine-tuned representations for task T. Such a neural network will almost always exist even if the fine-tuned representations have degraded significantly. Thus, it is a weak regular- izer and puts mild constraints on the change of the structure of representations.
B and Σn رلزز (B"B) B"y. y. Least square error can be written in terms of vector form to get
BT
So, V²
mEn Red
E(0 10) j=1
n² bb))
min vER
lly B
where 11.112 for a vector denote the & norm squared.
Substituting e² we get min
of b²bz)² - || BBB
LI
Theorem
1. We
show
that
D² o²=2l 20 tin) Fa-N((OIa)) mminDeRs Ej=(("' (u Zpre
)²
- (1 (m ( Z -nn) Zpre
12
Proof. To simplify notation, we use Oj sfreesj f/in B fin E R matrix has j-th row bj. A = Zyre RN matrix has j-th row aj and Xt is the pseudo-inverse of X.
Overall, this section can be summarized fol- lows: (i) Ê is an aggregate error in fitting fine-tuned representations to reeudo-pre-rranniie (ii) & controls the amount of structural changes in representations allowed during fine-tuning
tasks T, ss.
Let
N
B.1 Detailed Derivations
W = Eu-N(O,,a))
min UCR
(u
v
22 Fin
pre
Lemma 1. min.ERs Ej=( o- by)) 11v
B(B" By B" v112 where yj is the j-th entry of y.
From Lemma 1, we get
Proof. Let the loss function be
2
W Ea-N(O.IL)
Au B(B B) Au
N
C
Σ リj - b by)
Eu-N(O,Ta))
(A-B(B") B) BT A)u
Lemma 2. For any matrix M, Rdxd Eu) II) =|)
C is a smooth function with minimizer v². Hence, minimum is achieved at a local minimum. Thus,
|| where ||M] is the forbenius norm of the matrix M.
0 1=v = 0 80
Proof: Let the i,j-th entry of M be mijj and the j-th entry in u be uj. Then. LE mz
bリ (b*)Tbb)=
||Mu|
1=1
ΣS 1ド=1 El A= Ma
N
2
ربرر b,bद 0 =0
d
1=1
E Mull
=
Σ bj
yj
Σ b,bブ
0z
Since u is gaussian random variable with mean and covariance matrix Ia, we have Elu,ud] 0 for j f k and Elu?l = for all E [d]. Thus,
Σ 6o
bj
yj
E ||
2 m ||
where X the pseudo inverse which equal to the inverse if X is invertible. Else it spans only the space spanned by X. Note that Σ)= b,bj =

---

Substituting equality from Lemma 2 to W,
For j [N], [k], (i,i)thh entry of MV is m2 v. Thus,
get
2
W. = A A B(B" A
N M
Σ j=
(m]ee) v;)
Using - 1| and substituting back
A Zerr and B = fin we get Bo-N( minn ( 2
vf 2 fin )²
spre
Combining the two equalities, get
= (2 (Z 2 (im) Z In) Zpre
|
8 MM
Mu 2= |
Theorem 2.
(1 (2 (2 F(on)t 2 -nn) Zpre 1
N
Applying Lemma 3in eq 4, get
122
= =im WWRdxdd
pre
B(B" )A 2
L = [
Proof. To simplify notation, we use aj
This finishes the proof theorem.
zpreedy = =/ B fim RNxxd matrix has
j-th row bj, A Zpre R.d matrix has j-th row aj and X is the pseudo-inverse of X. Let W Radd have i-th row wy, We need to compute
= min WeRaad
Σ ag
Wbjl2
min "l...wgERdd
Σ
وjs ui b,)²
j=1 1=1
min Σ
(ays (ays - wi! by)
WRRRd
j=1
where ajs is the i-th entry of aj- Applying
Lemma 1, we get
Σ (In - B(B
(4)
where a is the i-th column of A (j-th entry of c; is aga)).
Lemma 3. For matrix M E NxN and set of vectors .,,..., Uk E N.N.
Mo = ||MVI
where V. Nxk is the matrix with columns
Ul...., Uke
Proof. Let j-th row of M be my. Then,
&
N
M Ug
Σ
Σ m 2)

---

C Experiment Set up details
C.2 Filtering Failed Runs
Our implementation is based on the HuggingFace library.
For most of the datasets experimented here, avail- able test data split is unlabeled. Thus, we use the validation data split to report performance. It has been observed that different finetuning runs can result in very different finetuned model per- formance (Mosbach al., 2020). Thus, reporting performance does not truly reflect the effectiveness the finetuning process and the max- imum test run performance across different random seeds can be substantially larger than the mean. So, in our experiments we do not use the val data (on which we report performance) to select the run or any hyperparameter. To select optimal hyperpa- rameters such regularization coefficient etc., subset of original train data split which is not used for training. Such a data available as we are typically finetuning with a subset of original train data. Table shows the threshold for failed run for each task.
C.1 Experimental Setup
To avoid the excessively high computational cost of fine-tuning on large-scale datasets, we limited their full training sets to 10, 000 data points (marked with suffix -10k in Table 3). For few-sample experiments, we fixed the same data subset across all models avoid performance changes related to data variability, Since test set labels are unavailable, we use development set to report the performance. Batch Size: Different methods have different mem- ory requirement. For instance, R3F has the highest footprint which limits the batch size as we can not process too many inputs the same time. Table 10 shows the batch size used for each dataset in our experiments.
max
Task Batch size
Task
Batch size
Task Threshold
Task
Threshold
COLA RTE SST MNII--OK MRPC
CHEMPROT SCICITE
COLA
0.00
CHEMPROT
34.45
SCITAIL-10k
RTE
53.70
SCICITE
24.66
AGNEWS-IOK
SST
54.00
SCITAIL 10k
60.38
YELP-10k
MNLI-10K
30.00
AGNEWS-IOK
10.44
QQP-10k
IMDB-I0K
MRPC
81.22
YELP-10K
52.80
QNLI-
QQP-1OK 0.00
IMDB-1OK
33.94
QNLI-10K
50.53
Table 10: Batch size used in our experiments
Table 11: Failed run threshold

---

D REPINAMLP- Effect of MLP depth and missing details
We use tanh activation in MLP with learning rate same as the rest of the network. Parameters of MLP optimized alongside the language model. We use Glorot uniform initializer initialize the parameters of MLP (Glorot and Bengio, 2010). Bias parameters are initialized to zero.
Missing Details:
Tasks
STD+
Top
Interm.
Training Datapoints
QNLI
80.86
80.7
82.56
MNLI
41.74
41.07
49.07
AGNEWS
89.09
89.34
89.53
Table 12 shows that REPINAMLP is resistant to the number of MLP layers chosen. When training with all datapoints, performance is typically within percentage of each other.
IMDB
83.65
89.77
91.32
SST
89.54
89.49
89.68
COLA
45.29
44.22
46.95
CHEMPROT
65.59
61.0
73.47
MRPC
84.44
84.54
85.26
SCITAIL
85.11
88.97
90.07
SCICITE
78.84
79.36
79.45
Task / Num. layers +
2
RTE
59.39
62.45
IMDB-10K
93.43
93.87
YELP
93.32
93.2
MRPC
91.19
91.12
QQP
61.12
69.5
70.97
SCICITE
82.55
83.15
Mean
73.81
74.34
77.23
COLA
61.86
62.47
Average Rank
2.38
2.54
1.08
SST
93.03
93.19
MNLI-1OK
65.18
65.22
Table 13:
Performance for REPINA -intermediate
AGNEWS-100
91.8
91.93
REPINAJ -top.
CHEMPROT
82.48
83.67
QNLI-10K
87.52
87.61
SCITAIL-ION
94.06
94.75
YRLP-1OK
95.66
95.96
RTE
72.92
74.37
QQP-10k
78.22
79.3
Mean
83.83
84,35
Table 12: Performance of REPINAMLP with different
number MLP layers.
E REPINA Which layer to regularize?
Table 13 compares the result between regularizing the top layer vs regularizing the intermediate layer REPINAJ We observe that REPINAJ consis- tently outperform when regularizing the intermedi- ate layer.
Tasks
STD+
5
10
20
500 Training datapoints
COLA
45.29
46.95
44.58
44.34
QNLI
80.86
82.56
81.38
80.7
MRPC
84.44
85.26
84.78
84.31
Table 14 shows the the result for REPINA with representations chosen from 5th, 10th 20th layer of encoder. Note that 5th layer is the closest to the input and doesn't 't account for token embedding layer. We note that all three choices are performing roughly equally well. Mean performance typi- cally less than percentage point from each other. If one were to use single layer, one can use 5th for low-data case and 10th or 20th for large dataset case.
SST
89.54
89.68
89.24
89.66
SCITAIL
85.11
90.36
90.07
89.2
YELP
93.32
92.88
93.2
93.2
AGNEWS
89.09
89.53
89.19
88,99
RTE
61.01
62.45
63.63
MNLI
41.74
44.61
49.07
44.57
QQP
61.12
72.6
71.79
70.97
IMDB
83.65
90.84
90.31
91.32
CHEMPROT
65.59
73.34
73.47
71.98
SCICITE
78.84
79.39
78.76
79.45
Mean
73.81
76.96
76.88
76.17
Table 14: Effect of embedding layer to be regularized in REPINA

---

F Hyperparameter Optimization over learning rate, epochs and dropout
Table 15 shows the results when we search over op- timal learning rate and number of epochs for each task and method. For learning rate, perform the search over [5e-6, (e-5,2e-5,4e-3 and for epochs, we search over [5, 10]. We add an additional base- line method where we search over dropout rate from [0.05, 0.1, 0.2, 0.4].
Tasks
STD+
DR
DA
WC Relnit
R3F
RP1
RPM
QQP
78.6
78.9
78.8
78.2
78.7
78.6
79.7
79.2
RTE
72.9
73.4
74.4
74.3
74.1
74.2
75.1
75.0
COLA
56.7
61.2
62.4
61.8
61.5
61.7
62.5
61.8
MRPC
90.3
90.8
90.5
90.4
90.8
90.4
90.8
90.8
Table 15: Results with HPO over epochs and learning rate, DR a baseline method where we do additional HPO over dropout rate well.
G Experiments for RoBERTa
In the results above, we observed that our meth- ods improve significant gain over baseline methods for BERT-large. Table shows the result when we compare REPINAJ against STD++.. We fine- tune the model for 10 epochs with regularization coefficient of 0.01 and learning rate le-5. Mean and standard deviation across three runs is reported. We observe that REPINAI improved STD++ perfor- mance in all cases.
Tasks
STD++
RPI
MRPC 90.3 + 1.0 92.0 + 0.5
RTE 74.1 2.1 77.1 + 0.7
CoLAA
60.0 1.1 60.1 + 0.6
Table 16: Results for RoBERTa-base on 3 GLUE datasets.

---

H Supervised Contrastive Learning
Tasks
STD+ SCL REPINAI REPINAMLP
250 datapoints
Let mini-batch has examples, (01, ....... ... (mmm Dm) and 21,...; Fm be the representations (output of encoder) using the model being finetuned. Supervised Contrastive Learning encourages the representations of examples of same label in the mini-batch to be close to each other and far from the examples with different label by additing the following loss to the objective:
QNLI
75.11
73.79
78.82
76.13
SST
88.41
87.27
89.29
88.59
QQP
68.28
68.79
68.76
70.38
SCITAIL.
82.31
86.13
88.9
86.5
MNLI
37.7
38.07
38.35
41.18
IMDB
86.11
90.27
90.38
90.33
RTE
59.13
60.77
60.83
59.3
MRPC
84,43
85.67
84.65
84,08
COLA
41.57
31.91
43,98
45,28
CHEMPROT
55.22
32.0
63.28
62.32
Mean
67.83
65.47
70.72
70.41
Average Rank
4.3
3.7
1.6
2.35
m Σ 1=1
m 1 Lips' yル
Table 17: Performance memory-efficiet SCL.
LSCL =
NA.
j=1
lowing relaxation of LSCL.
cocp( Zi, 2)) k= lppccpp (zi, 2x)/T)
log
N
1
LSCL
ligj-u=y
N
where T is a scalable temperature parameter that controls the separation of classes. Loss function during training is
expp(zz cps)/T)
log
k= Lipneappl (Zi), Cyy ) /T)
L - + (1- X)CsCI.
j
j=1
(2, c/)TT
where Loe is the cross entropy loss where À is factor that be tuned. This was shown to improve finetuning process (Gunel et al., 2020) for few-shot finetuning.
log
1k Ca (xlepp((zz (t)
A naive implementation of this loss function would be very expansive as the centers Cl...., would change in each iteration. We observe that centers change much slower than the individual examples. This is the reason to replace individual training samples with the centers while comput- ing similarity (24, 2z)- Thus, we do not update each iteration and instead update it only ten times during the finetuning process. Note that it increases the training time by roughly a factor of 10 which is also prohibitive for large datasets. Ta- ble 17 shows the comparison of memory efficient SCL with our methods. We that both REPINA and REPINAMLP beat SCL consistently More- over, SCL incur significant loss for several datasets.
From the definition of LSCL observe that SCL is only effective when the mini-batch size is large and each label class sufficiently represented in the mini-batch. Otherwise, the loss function ESCL is vacuous. For instance, if the mini-batch size is 1 which is the case for many of our datasets, then LSCL 0 for all the mini-batches. Thus, is equivalent to the standard finetuning. Large mini- batch size however requires large memory during finetuning process which is not always available as in our case. Thus, we look for a relaxation of SCL which can be implemented manner.
memory efficient
We start by considering LSCL over the entire input set instead of mini-batch and then replace the example 2j with mean of examples of the same class as Zy while computing similarity with an- other example. More formally, let the training data be (11, (ZN, Bn), the set of labels be (1..... f) and representation of x; from the en- coder of finetuning model. Let C₁- {il y - and 6 J Evec, 2j be the center of embed- dings of inputs with label j. We consider the fol-

---

I Comparison of R and REPINAMLP against each baseline
Table 18 show that both REPINAi and RE- MLP outperform each baseline method in ma- jority of the datascts/
REPINA1 REPINAMLP
wins against baselines methods
GLUE datasets (out of
STD+
7
6
DA
5
5
WC
5
6
Relnit
7
7
R3F
7
7
Non-GLUE datasets (out of 6)
STD++
6
WC
3
5
Relnit
6
R3F
6
6
Table 18: Number of tasks which REPINAJ or
PINAMLP outperform the baseline method.
J Small dataset results
Table 19 and 20 show the performance for few- sample finetuning setting.
Tisks
REPINAJ
REPNNAMI PP
Training thtapuints
QOP
68.28
65.83
69.01
68:46
67.45
68.76
70.38
COLA
41.57
40.39
45.14
43.73
40.76
43.98
45.28
RTE
59.13
61.37
$3.99
60.29
60.02
60.83
59.3
MNLI
37.7
40.24
33.71
37.8
38.35
41.18
YELP
93.3
ONLI
75.11
76
IMDB
86.11
89.17
89:51
89.44
89.09
9IL38
90.33
SCICITE
76.36
77.2
76.23
78,56
77.30
76.11
75.76
SST
88.41
88.1
$5.11
65.3
88.47
89.29
83.59
MRPC
84.43
83.81
84.93
84.87
84.71
84.65
84-08
SCITAIL
82.31
87.65
87.8
88.9
86.5
AGNEWS
88.13
71.98
Average Rank
3.62
4.92
4.62
4,04
2,5
3.31
Table 19: Performance for different regularization meth-
ods.
Tasks
STD++
DA
WC
Relnit
RJF
REPINA)
REPINAMI
500 Training dacapoints
QQP
COLA
45.29
15.36
16.76
XTE
61.01
60.53
61.25
63.33
60.41
62:45
62.94
MNLI
41.34
49.05
46.32
51.11
42.25
19.07
14.73
YELP
95.32
93.46
92.87
93.04
93.27
93.2
93.35
CHEMPROT
65.59
74.2
70.78
73.2
70.04
73.47
72.9
80.86
K2.43
IMDB
83.65
SCICITE
80.4
MRPC
86:44
$4.69
$5.4
$1.93
81.77
85.26
85.62
SCITAIL
85.11
87.41
90.23
80 18
80.73
90.07
80.3
AGNEWS
89.09
88.93
$9.0
$9.09
89.59
99.53
89.54
Mein
23.81
76.72
76.63
77.24
75.26
77.23
76.84
2.92
Table 20; Performance for different regularization meth-
ods.

---

K Results without any filtered runs
STD+
DA
WC
Relnit
R3F
RPI
RPM
Successful runs (Failed runs filtered)
Mean
81.16
83.74
83.58
83.3
82.55
84.01
84.36
2.87
0.56
0.83
0.76
2.19
0.61
0.49
Frac
0.34
0.06
0.18
0.05
0.11
0.00
0.04
All runs (Failed runs not fittered)
Mean
64.71
78.58
81.4
79.97
72.97
33.99
83.04
std
13.4
6.14
2.89
5.89
8.62 0.60
1.28
Table 21: Stability of fine-tuning results. RP RE- PINAI RPM REPINAMLP Frac: fraction of fine- tuning runs filtered due to low performance. Mean, std: mean and standard deviation in performance across all datasets.
Table 22, and 23 shows performance without filtering out failed runs.
Tisks
STD+
DA
WC
Reinit
R3F
REPINAT
REPINAME.S
250 datapoints
YELP
92.51
92.68
92.97
93.64
93.13
93.13
93.57
RTE
55.6
58.12
55.96
60.29
58.88
59.78
60.14
MNLI
25.75
30.22
28.42
41.47
31.94
39.2
38.58
72.71
SCITAIL
82.31
65
86.2
87.8
88.6
88.9
86.5
SST
88.41
88.1
88.
88.3
88.47
89.29
88.59
CHEMPROT
55.22
58.6
59.52
64,92
59.6
60.72
62.32
AGNEWS
88.08
87.51
87.55
87.36
87.99
88.53
88.13
SCICTTE
76.86
77.2
76.23
78.86
77.39
78.95
75.76
IMDB
83.5
80.13
89.51
89.44 44
89.05
90.38
90.33
41.57
40.39
45.14
43.73
40.76
43.98
45.28
84.43
84,93
84.71
84.65
84.08
35.94
68.76
65.5
67.91
74.24
73.45
Average Rank
5.92
5.46
4.85
3.88
1.96
3.23
Performance for different regularization meth- Table 22:
ods without filtering failed runs.
Tasks
STD++
ReInit datapoints
REPINAJ
REPINAMLP
MRPC
86.84
91.12
IMDB-10K
66.59
93.24
93.69
92.7
93.1
93.96
93.87
YEI PO1
72.65
95.55
95.81
95.56
95.42
78
95,96
SCICITE
81.87
82.13
82.23
82.41
81,97
82.74
83,15
QNLI 10k
64.93
81.64
86.1
86.73
78.43
86.85
87.36
CHEMPROT
72.59
82.57
83.91
82.46
82.73
83.49
83.67
MNLI--OK
14.37
45.98
55.94
46.54
21.66
65.48
64.81
COLA
59.69
63.45
62.04
62.34
62.47
RTE
51.35
71.26
AGNEWS-108
91.67
92.07
91.93
SST
81.95
84.13
92.71
93.23
QOP- 10k
5.9
47.77
76.25
55.16
27.8
79.03
79.3
SCITAIL- 10K
76.01
99.774
94.03
93.36
86.54
93.74
94.75
Mean
63.57
77.6
81.17
79.84
73.22
83.92
83.31
Average Rank
6.92
4.38
3.46
4.38
5.31
1.92
1.62
Performance different regularization meth- Table 23:
ods without filtering failed runs.

---

L Detailed results for label noise
Table 24, 25, 26 shows detailed results with varying amount of label noise in the training data.
Tasks
STD+
DA
WC
Refnit
R3F
REPINAT
REPINAMIP
Noise level
0.05
SCICITE
81.29
56.46
81.34
81.11
81,03
81.36
81.88
QNLI-IOK
57.35
50.18
68.59
67.16
64.33
83.87
86.08
MRPC
87.4
87.48
86.66
89.11
88.41
88.41
86.81
RTE
51.35
48.59
51.55
65.63
48.65
67.58
61.16
IMDB-10K 10K
72.43
68.28
77.4
62.14
91.63
92.84
92.56
CHEMPROT
71.57
82.18
AGNEWS-I00
91.1
91.26
91.21
91.47
YELP-1OK
49.92
72.44
86.24
85.4
72.38
95.1
95.36
MNLI-1OK 10k
0.0
0.0
31.9
41.99
47.51
63.26
24.96
SST-10k
91.12
83.17
91.83
90.05
91.31
83.21
90.73
SCITAIL- 10K
58.21
57.94
92.91
83.36
83.93
92.32
93.48
QQP-1OK
0.0
0.0
76.24
57.38
0.0
77.78
78.82
COLA
46.38
47.84
59.33
43.47
45.15
59.53
48.29
Mean
58.32
57.3
75.34
72.28
68.24
81.39
77.98
Average Rank
5.5
5.96
2.92
4.35
2.42
2.46
Table 24: Training with at most 10k training datapoints on 13 datasets.
Tasks
STD++
ReInit
REPINAT
REPINAMLP
Noise lovel =
SST-1Ok
82.64
MNLI-IO
12.44
0.13
57.55
48.92
24.31
59.67
12.38
SCITAIL-100
65.86
63.01
74.68
80.92
69.86
90.97
92.82
IMDB-10K
33.33
33.33
44.86
67.47
76.92
90.96
91.88
RTE
50,54
49.46
48.38
61.52
50.54
64.77
60.58
MRPC
84.11
83.69
85.11
88.49
84,33
86.06
87.4
AGNEWS- 10k
90.19
90.24
90.54
90.23
90.32
90.51
90.5%
CHEMPROT
80.75
68.56
80.56
69.38
$2.49
71.92
QQP-10k
0.0
14.92
76.17
76.76
YELP-1OK
60.78
94,22
94.64
COLA
56.75
45.85
58.65
52.8
45.43
44.12
55.6
QNLI-IOK
50.32
50.54
65.61
63.15
66.69
74.24
83,83
SCICITE
80.79
69.21
80.45
80.71
80,29
78.76
81.52
Mean
57.99
52.44
72.22
68.86
63.68
78.13
75.58
Average Rank
4.96
6.46
3.23
3.46
4.73
2.77
2.38
Table 25: Training with most 10k training datapoints
13 datasets.
Tasks
STD+
DA
Relnit
R3F
REPINAJ
REPINAMLP
Noise level D.3
QQP-10K
0.0
0.0
68.62
13.08
16.01
68.31
67.36
SCICITE
34.18
24.67
43.99
73.7
62,73
67.3
73.7
COLA
3.42
6.67
35.97
12.8
15.01
24.99
IMDB-
33.33
33.33
35.6
MNLI-1Ok
0.0
0.22
0.0
CHEMPROT
41.96
54.1
69.21
75.81
45.41
67.7
$3.96
QNLI-IOR 10k
50.11
49,89
$6.27
55.13
50,27
50. 18
77.96
AGNEWS-II 10k
70.31
83.14
67.91
81.75
82.41
45.87
68.06
RTE
49.46
49.46
51.48
53.14
49.46
60.83
54,51
YELP-10K 10k
50.02
49.56
56.02
60.22
49.79
51.99
91.44
SCITAIL-1OK 10K
49.62
49.62
58.45
49.62
49.62
56.83
86.71
MRPC
$0.92
79.54
80.63
SST-10k
58.23
64.04
57.73
75.53
63.03
57.08
57.98
Mcan
40.12
52.44
50.32
59.45
Average Rank
5.5
4.88
3.23
4.54
4.04
3.04
Table 26: Training with at most 10k training datapoints datasets.

---

M Representation Collapse Continual learning perspective
Table 27, 28, 29 shows results for representation collapse when we finetune the model for task A using different methods and then finetune the top layer for task B.
Tasks
SID++
DA
WC
Reinit
R3F
REPINA
REPINAMLP
COLA
-0.38
0.63
3.39
-1.17
0.93
5.08
MRPC
81.62
81.22
83,13
81.57
81.44
82.12
82.42
QQP-IOK
29.68
31.71
YELP-10K
50.86
51.55
55.13
52.63
60.04
SCITAIL-IOK
59.55
58.44
74.46
61.32
49.62
67.38
71.24
SCICITE
24.82
24.84
28.74
24.75
24.67
25.02
27.71
AGNEWS- 10k
20.22
20.5
36.61
16,92
20.15
31.01
62.9
IMDB-10 10k
41,36
33.33
49.87
41.0
43.36
43.57
55.72
CHEMPROT
33.1
32.46
33.23
32,92
32.34
33.09
33.15
MNLI-IOK
8.07
8:27
16.76
8.36
6,92
14.58
17.69
SST-1OK
51.25
54.55
58.21
52.24
52.2
55,93
66.36
51.5
53.55
37.62
37.69
Average
1,92
5.67
3.33
Table 27: Results for training top layer for different task after finetuning entire model for QNLI-10K
SID++
Reluit
R3F
REPINA
REPINAMIL
81.45
82.67
CHEMPROT
33.46
33.46
33.46
33.46
33.46
33.59
33,46
QNLI-10K
59.06
63.74
67.98
61.65
50.54
64.95
64,58
YELP-10K
50.18
50.02
55.91
51.43
50.06
64.07
51.32
62.54
72.01
68.1
65.55
49.62
76.55
71.55
COLA
0.23
-0,79
0.0
-0.01
0.0
5.17
-0.82
AGNEWS- 10k
16.02
19.77
32.56
19.6
10.76
64.32
38.5
IMDB-IOK
39.92
45.49
39.17
33.33
66.28
SCICITE
24.67
24.67
24.67
24.67
28.51
MNLI-IOK
11.73
19.57
SST-1OK
49.69
52.24
50.89
51.03
69,87
58.08
RTB
48.86
52.71
52.89
51.44
50.9
53,52
52.17
Mean
39.82
41.79
44.57
41.23
36.3
52.38
44,05
Average Rank
4.96
3.79
3.08
4.79
5.67
1.17
4.04
Table 28:
Results for training top layer different task
after finetuning entire model for QQP-10k
Tasks
STD+
DA
WC
Relnit
R3F
REPINAT
REPINAMLP
COLA
0.0
0.0
1.48
-0.59
0.0
5.06
1.91
MRPC
81.22
81.22
81.31
80.75
81.22
81.68
81.91
QQP-10k
0.0
0.0
18.89
60.51
0.0
60.87
36.29
QNLI-1OK
50.54
50.54
57.24
58.24
50.54
62.8
55.56
YELP-10K
50.02
50.02
50.15
58.96
53.13
63.63
57.1
SCITAIL--OO
49.62
49.62
AGNEWS
21.44
IMDB-10K
33.33
33.33
38.1
56.09
35.22
$7.18
44,89
SCICITE:
24.67
24.67
25.14
25.33
24.67
31.81
25.96
CHEMPROT
33.46
33.46
33.46
33.26
33.46
33,56
33.46
MNLI-100 10k
0.0
0.0
6.4
29.24
2.67
27.0
20.51
SST-10K
50.92
50.92
55.59
60.92
53.36
63.88
56.86
Mean
31.98
31.98
37.24
48.85
33.25
51.85
41.96
Averuge Rank
5.96
5.88
3.75
3,25
5.08
1.17
2.92
Table 29; after finetuning entire model for RTT:
Results for training top layer for different task

---

N Measuring representation collapse
Table 30 show the sum of top-k normalized eigen- values (divide each eigenvalue by the sum of eigen- values) for k=10. From this, we can observe that almost all the normalized eigenvalues after the first twenty are close to zero
Tasks
STD+
DA
Relnit
R3F
REPINAI
REPINAMLP
RTE
1.0
1.0
1.0
0.99
1.0
0.99
MRPC
1.0
0.99
0.99
0.96
1.0
SCITAIL-110
1.0
1.0
1.0
1.0
1.0
IMDB 10K
1.0
1.0
1.0
1.0
1.0
SST-1Ok
1.0
1.0
0.99
1.0
1.0
1.0
1.0
COLA
0.99
0.99
0.98
0.99
1.0
1.0
1.0
AGNEWS-IOK
0.98
0.98
0.98
0.99
0.98
0.98
0.99
QQP 10K
1.0
1.0
1.0
1.0
MNLI-IOK
0.99
0.99
CHEMPROT
0.98
0.97
0.97
0.98
SCICITE
0.99
0.99
0.98
1.0
1.0
0.98
0.99
Table 30: Normalized average of top- 10 eigenvalues
Tables 31 and 32 show the GM-k and HM-K for k=10. We observe that REPINAJ achieves the highest value and thus is most effective in reducing representation collapse.
Tisks
STD+
DA
WC
Relnit
R3F
REPINAT
REPINAMLP
RTE
0.01
0.01
28 58
54.68
0.54
113.27
24.64
MRPC
47.83
85.21
108.67
65.41
59.4
61.13
19.25
QNLI-IOK
29.3
34.28
54.86
16.92
29.65
47.15
46.11
SCITAIL- 10k
25.23
58.43
47.44
26.62
31.48
49.61
33.93
IMDB-BE 10k
32.82
32.53
29.47
12.69
23.31
48.51
35.45
58.83
COLA 69.32
82.06
84.79
42.05
68.46
AGNEWS-
220.03
221.59
186.54
202.
208.1
179.2
QQP- 10k
10.5
25.4
64.85
22.53
0.01
90.66
35.88
MNLI-IOK
20.29
82.66
46.99
115.1
34.8
164.62
71.13
YELP-10K
19.27
33.91
39.9
8.13
10.12
33.49
25.33
CHEMPROT
499.58
646.59
613.5
603.17 17
601.67
619.9
446.36
SCICITE
164.7
171.53
163.86
88.84
108.6
190.12
152.38
Mean
92.13
118.64
118.04
90.89
93.2
134.96
91.2
Average Rank
5.08
2.62
2.85
5.38
5.31
2.31
4.46
Table 31: GM-10
Tasks
STD+
DA
WC
Reinit
R3F
REPINAT
REPINAMI
RTE
0.0
0.0
9.0
14.79
0.1
41.73
7.15
MRPC
15.09
24.73
33.48
26.9
13.53
17.29
3.31
QNLI-1OK
6.99
9.65
16.49
3.13
7.1
11.6
12.37
SCITAIL- 10k
5.46
14.88
14.11
4.35
6.74
10.75
7.91
IMDB-10K 10k
6.7
6.83
7.81
1.7
4.09
11.92
6.44
SST-1Ok
15.31
COLA
19.72
24.21
28.76
9.88
19.34
20.15
AGNEWS- 10k
65.88
67.33
54.52
18.78
60.41
62.36
44.45
QOP- 10k
1.56
5.6
17.67
3.68
0.0
26.38
9.33
MNLI-IOT 10k
5.68
27.28
13.1
28.45
10.77
$I.47
20.74
YELP-10 10k
3.28
7.42
11.0
0.95
1,88
7.14
4.95
CHEMPROT
302.28
429.53
333.57
356.82
332.11
389.25
226.06
SCICITE
48.68
54.7
56.06
18.57
29.54
$9.65
45.86
Mcan
38.2
53.19
47.44
37.99
38.04
55.7
32.16
Average Rank
5.08
2.77
2.31
5.31
5.46
2.46
4.62
Table 32:
10

---

Walltime Analysis
Walltime analysis: STD+ uses a single forward and backward pass with simplest loss function and thus has the least training time. Relnit is a close second as it only differs in the initialization of the model. WC also uses single forward and back- ward pass but is slower due to the regularization loss function computation. R3F and DA use two forward passes and two (effective) backward passes. Our method on the other hand use only one for- ward and backward pass. In addition to that we use only an extra forward pass of the pretrained model. Thus, our method is slower than STD++ ReInit and WC and is faster than R3F and DA. Table 33 show the training time for all the methods. We observe that R3F consistently takes more time than all the methods. REPINA runs faster than R3F and DA but slower than STD++ WC and Relnit. REPINAMLP runs slower than REPINAI
Tasks
STD++
DA
WC
ReInit
R3F
REPINA
REPINAMLP
CHEMPROT
584.01
1015.37
702.38
589.39
1415.28
826.09
114166
MNLI-10K
1506.74
2643.21
1723.52
1514.14
3746.99
1832.83
3022.97
SCITAIL-10R
1678.89
2962.31
1896.3
1684.06
4217.93
2418.12
3130.63
MRPC
162.09
258.92
214.94
157.42
362.83
226.59
299.13
QNLI-1OK
1683,58
2966.56
1894.97
1681.34
4232.07
2414.78
3124.71
QQP-10k
1272.46
2194.95
1487.52
1276.92
3078.56
2307.66 62
2302.41
SST
5050.32
8537.84
5997.04
5080.05
11878.49
6019.1
9568.79
COLA
239.78
375.78
349.94
240.61
515.27
292.32
398.9
SCICITE
961.03
1746.23
1066.78
962.4
2838.07
1850.68
1835.6
IMDB-10 10k
1679.74
2975.29
1899.58
1692.33
4220.9
3149.3
3124.5
YELP-10K
1681.65
2975.09
1894.85
1686.21
4224.52
2426.62
3130.62
307.13
504.8
396.35
309.26
698.16
527.76
$29.66
AGNEWS-IOK
589.23
1046.27
675.37
595.06
1449.9
1125.27
1227.09
Mean
1338.2
2323.28
1553.81
1343.78
3298.38
1955.16
2525.9
Table 33: Training time for different methods
P Connection of GMand HMto parameter estimation error
Let the pseudo linear regression task finetuned representations be defined by E Rd and the noisy labels observed on 2,'s yi = 2/" t where e's are the gaussian noise centered around 0 with iden- tity covariance matrix. ω is the least square min- imizer (same as log-lik ligllikll maximizer), then à w + N 1-))
GM corresponds to minimizing the confidence cllipsoid corresponding to the eπror ④ HM corresponds to minimizing the expected 6 norm of the error vector u w. Derivation of the ú and the explanation can be found in Madan et al. (2019).


Reviews:
[{"review_id": "RMm474Pyzp", "reviewer": "Reviewer_HLED", "summary": "", "questions": "1. line 130-138: I don’t understand the logic of “regularization loss would be zero … however … regularization loss would be high”. The writing could be improved. \n2. For the OOD experiments in Section 5.3, using a bar chart to show generalization gap as in Hendrycks et al. (2020) would make the results clearer. The current table format is difficult to parse for assessing OOD robustness.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* This paper proposes a novel regularization approach targeting representation collapse during fine-tuning, complementing existing techniques that regularize model weights/gradients.\n* The experiments are well-structured, extensive, and solid, covering OOD evaluation, label perturbation, probing representation collapse, representation diversity, etc.\n* The paper is generally well-written and easy to follow.", "reasons_to_reject": "* This paper only compares the proposed methods with models published before 2021. It lacks discussion of recent parameter-efficient fine-tuning strategies like fixed sparse masking and low-rank adaptation.\n* Section 2 claims REPINAI allows stronger regularization than REPINAMLP but provides no direct experimental evidence supporting this intuition.\n", "questions_to_authors": "1. line 130-138: I don’t understand the logic of “regularization loss would be zero … however … regularization loss would be high”. The writing could be improved. \n2. For the OOD experiments in Section 5.3, using a bar chart to show generalization gap as in Hendrycks et al. (2020) would make the results clearer. The current table format is difficult to parse for assessing OOD robustness.", "paper_topic_and_main_contributions": "This paper proposes a new regularization method called Representation Projection Invariance (REPINA) to mitigate representation collapse during fine-tuning of pretrained language models. The key idea is to regularize the fine-tuned representations to be invariant to projections by a chosen set of functions. The paper presents two versions of REPINA – REPINAI uses an identity function for projection and REPINAMLP uses a multilayer perceptron, which allows applying stronger or milder regularization.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "* (2021) Training Neural Networks with Fixed Sparse Masks\n* (2021) LORA: LOW-RANK ADAPTATION OF LARGE LAN- GUAGE MODELS"}, "extra_text": {}}, {"review_id": "p0ccZx5vtj", "reviewer": "Reviewer_E5eQ", "summary": "", "questions": "* Could the authors provide the eigenvalues of the presentations for both unseen and seen tasks in the pre-trained model? It would be beneficial if Figure 5 could also illustrate the anticipated eigenvalues.\n\n* Is there empirical validation supporting the selection of layer values as 5, 10, or 20? \n\n* What is the mean performance (accuracy) for Table 5?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* One commendable aspect of the paper is its consistent achievement of state-of-the-art performance across multiple datasets. This is a testament to the efficacy of the proposed REPINA methodology.\n\n* The authors' inclusion of comprehensive ablation studies contributes significantly to the clarity and depth of their findings. These studies enhance our understanding of the tangible impact REPINA can have on the fine-tuning process.\n\n* The paper underscores its merit by demonstrating the superiority of REPINA over other baseline methods in various robustness scenarios, including out-of-distribution settings, label perturbations, and few-shot fine-tuning. This underlines the versatility of the proposed approach.\n\n* The paper is well written.", "reasons_to_reject": "While the paper demonstrates impactful contributions to the field, there are certain aspects that merit further attention:\n\n* One notable omission is the absence of an essential baseline, namely LoRA for LMs (Low-Rank Adaptation) [1], which could offer a meaningful point of comparison. \n* Additionally, expanding the experimentation to include diverse models beyond BERT or RoBERTa, such as T5 or GPT-2, would strengthen the paper's conclusions.\n\nNote: While it is acceptable to concentrate ablations or analyses on one model, I recommend diversifying the model selection and adding similar tables as Table 1 for different models. Including experiments across a wider array of models would provide a more comprehensive perspective on the methodology's applicability.", "questions_to_authors": "* Could the authors provide the eigenvalues of the presentations for both unseen and seen tasks in the pre-trained model? It would be beneficial if Figure 5 could also illustrate the anticipated eigenvalues.\n\n* Is there empirical validation supporting the selection of layer values as 5, 10, or 20? \n\n* What is the mean performance (accuracy) for Table 5?", "paper_topic_and_main_contributions": "The paper addresses the critical issue of representation collapse during fine-tuning of language models for downstream tasks. The authors propose a regularization-based approach called REPINA to maintain the expressivity within latent representations throughout the fine-tuning process. The evaluation encompasses 13 datasets, along with comparisons against five distinct baseline methods.", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "[1] Hu, Edward J., Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. \"Lora: Low-rank adaptation of large language models.\" arXiv preprint arXiv:2106.09685 (2021)."}}, {"review_id": "6KGtvKtouA", "reviewer": "Reviewer_4e8F", "summary": "", "questions": "How do you envision extending the insights gained from REPINA to other aspects of fine-tuning, such as transfer learning, multi-task learning, or domain adaptation, where representation collapse could also play a role?\n\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1、The paper addresses a significant challenge in NLP – representation collapse during fine-tuning – and presents a novel solution, REPINA. This issue is particularly relevant given the widespread use of pre-trained language models.\n\n2、The extension of previous studies by introducing metrics to quantify representation collapse enhances the paper's contribution to the field. This added insight provides a valuable framework for analyzing and addressing this issue.\n\nThe demonstrated effectiveness of REPINA in in-domain, out-of-distribution, few-shot settings, and robustness to label perturbation highlights its versatility and potential applicability across a range of NLU scenarios.", "reasons_to_reject": "1、The paper mentions that REPINA consistently outperforms baselines on most tasks. However, it could provide a more detailed comparison, including specific performance metrics and a deeper analysis of why REPINA performs better.\n\n2、The author should explore more about the applicability of the method in this paper to prompt tuning settings and LLMs.", "questions_to_authors": "How do you envision extending the insights gained from REPINA to other aspects of fine-tuning, such as transfer learning, multi-task learning, or domain adaptation, where representation collapse could also play a role?\n\n", "paper_topic_and_main_contributions": "This paper addresses the issue of representation degradation (representation collapse) that can occur during fine-tuning of pre-trained language models in NLP. It introduces a novel regularization method called Representation Projection Invariance (REPINA) to mitigate representation collapse by discouraging undesirable changes in representations while fine-tuning. The study empirically compares the proposed method with five comparable baselines across 13 language understanding tasks, including the GLUE benchmark and additional datasets. The results demonstrate REPINA's consistent outperformance of other baselines on most tasks, both in-domain and out-of-distribution. The paper also presents the extension of previous studies on representation collapse by introducing metrics to quantify it.\n\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7Xlyw7XcqJ", "reviewer": "Reviewer_DTwA", "summary": "", "questions": "1. Would it be possible to also report the OOD results for RP_{MLP} in Table 3 ?\n2. Would it be possible to also consider the motivating set of linear projections as a part of the experiments ? Since it is a linear projection, the solution to the inner minimum can be obtained analytically at each step of the optimization process.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed method, while simple (especially REPINA_{I}) demonstrates strong performance and low levels of representational collapse compared to other baseline approaches\n2. The experiments are quite comprehensive, across a diverse number of datasets and compared against a wide variety of baselines\n3. The connection to multi task learning is quite intriguing, and merits additional exploration\n4. The ablation studies w.r.t OOD robustness, robustness to label perturbation as well as the experiments measuring representational collapse underscore the ability of the proposed method in preserving the pre-trained models representational space.", "reasons_to_reject": "1. The setup of the MLP formulation is a bit underspecified: is the min actually solving an inner optimization problem (so at every training step, the MLP minimized (via multiple steps of gradient descent on the MLP's parameters), or is it a first order approximation (so only one step of minimization for each training step). Is there a performance delta between the two approaches ? For the first one, the wall-clock time would substantially increase (because there would be multiple gradient descent steps on the MLP's parameters per optimization step). \n\n2. Some of the baselines omit stronger counter-parts. For example, weight consolidation does not consider the EWC variant that upweighs feature importance based on the the approximate fisher information (eg: see [1]). That has been shown to perform better than vanilla EWC. Likewise, Sharpness Aware Minimization techniques have been demonstrated to perform better than R3F and R4F ([2]). Hence those should be used to establish the baselines, in my opinion.\n\n3. The proposed Diversity of Fine-tuned representations methodology for understanding representational collapse requires additional explorations. Specifically, correlation studies between the proposed method and linear probes, as well as stability while using across different datasets needs to be quantified before the metric can be adopted as another way for checking for representational collapse. In addition to that, the metric (to my understanding) is analogous to looking at the variance of datapoints along the principle components of the representations. Thus, there are cases where this would not be able to detect a representational collapse (eg: say the collapse causes the space to be uniformly spread along a sphere, as opposed to occupying the entire space. In that case, there is a collapse, but this would not be detected by the proposed method). At the very least, the stability across different datasets and correlation metrics would inspire some level of confidence for the proposed method.\n\n\n[1] Thorne, James, and Andreas Vlachos. \"Elastic weight consolidation for better bias inoculation.\" arXiv preprint arXiv:2004.14366 (2020).\n[2] Bahri, Dara, Hossein Mobahi, and Yi Tay. \"Sharpness-aware minimization improves language model generalization.\" arXiv preprint arXiv:2110.08529 (2021).", "questions_to_authors": "1. Would it be possible to also report the OOD results for RP_{MLP} in Table 3 ?\n2. Would it be possible to also consider the motivating set of linear projections as a part of the experiments ? Since it is a linear projection, the solution to the inner minimum can be obtained analytically at each step of the optimization process.", "paper_topic_and_main_contributions": "The paper presents a new regularization method: representation projection invariance regularization dubbed REPINA. Specifically, the authors\n\n1. Present a new regularization term that is based on the distance between the pre-trained representations and a transformed version of the fine-tuned representations, where the class of applicable transformations determines the strength of the regularization.\n2. Draw a connection between their proposed method and a multi-task setup with infinite pseudo tasks.\n3. Demonstrate on 13 tasks the effectiveness of their proposed approach compared to other baselines for 10 / 13 tasks\n4. Show that REPINA allows for higher run stability, OOD performance, Robustness to label noise and better performance on the few-shot fine-tuning setup\n5. Demonstrate that the proposed method shows less representation collapse when measured using probing experiments\n6. Propose a new method to quantify dimensional collapse based on the eigenvalues of the Gram matrix, showing that REPINA still shows least amount of representational collapse compared to other metrics.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Zheng, Bo, et al. \"Consistency regularization for cross-lingual fine-tuning.\" arXiv preprint arXiv:2106.08226 (2021) might be relevant in the context of data augmentation based regularization (L215)\nBahri, Dara, Hossein Mobahi, and Yi Tay. \"Sharpness-aware minimization improves language model generalization.\" arXiv preprint arXiv:2110.08529 (2021) might be applicable for general purpose fine-tuning approaches.\n", "Typos_Grammar_Style_and_Presentation_Improvements": "L130: we bias \"fine-tuned representations to be linear transformations of the fine-tuned representations -> we bias pre-trained representations to be linear transformations of the fine-tuned representations ?\nL133: Thus, regularization loss in case of Fig 2a would be zero -> Thus, regularization loss in case of Fig 2b would be zero ? "}}, {"review_id": "3PsGLogIhI", "reviewer": "Reviewer_1NmN", "summary": "", "questions": "- In line 354, does the \"fail run\" filter applied to all of the baselines in your comparsons?\n- In terms of REPINA's performance, it appears to lag behind the other baselines in certain tasks. Do you have any insights regarding the specific reasons behind these shortcomings? ", "limitations": "", "rating": 3, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed Idea is clear and effective, leading to consistant improvement in fine-tuning\n- The paper is well written\n- The experiments and analysis are comprehansive\n", "reasons_to_reject": "- The paper only perform with BERT-large model, leaving uncertainty regarding the applicability of the proposed approach to other pre-trained model architectures.\n", "questions_to_authors": "- In line 354, does the \"fail run\" filter applied to all of the baselines in your comparsons?\n- In terms of REPINA's performance, it appears to lag behind the other baselines in certain tasks. Do you have any insights regarding the specific reasons behind these shortcomings? ", "paper_topic_and_main_contributions": "The paper introduces a novel regularization method, REPINA, designed to address representation collapse during the fine-tuning of pre-trained language models. The study evaluates REPINA against five comparable baselines across a diverse set of 13 language understanding tasks, and find REPINA consistantly outperform other baselines on most of them.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a novel regularization approach targeting representation collapse during fine-tuning, complementing existing techniques that regularize model weights/gradients. The experiments are well-structured, extensive, and solid, covering OOD evaluation, label perturbation, probing representation collapse, representation diversity, etc. The paper is generally well-written and easy to follow. However, this paper only compares the proposed methods with models published before 2021. It lacks discussion of recent parameter-efficient fine-tuning strategies like fixed sparse masking and low-rank adaptation. Besides, the paper mentions that REPINA consistently outperforms baselines on most tasks. However, it could provide a more detailed comparison, including specific performance metrics and a deeper analysis of why REPINA performs better.
Decision: Accept-Findings