Fine-tuning can cripple your foundation model; preserving features may be the solution
Authors: Jishnu Mukhoti
Yarin Gal
Philip Torr
Puneet K. Dokania
Venue: Accepted by TMLR ()

Abstract: Pre-trained foundation models, due to their enormous capacity and exposure to vast amounts of data during pre-training, are known to have learned plenty of real-world concepts. An important step in making these pre-trained models effective on downstream tasks is to fine-tune them on related datasets. While various fine-tuning methods have been devised and have been shown to be highly effective, we observe that a fine-tuned model's ability to recognize concepts on tasks different from the downstream one is reduced significantly compared to its pre-trained counterpart. This is an undesirable effect of fine-tuning as a substantial amount of resources was used to learn these pre-trained concepts in the first place. We call this phenomenon "concept forgetting'' and via experiments show that most end-to-end fine-tuning approaches suffer heavily from this side effect. To this end, we propose a simple fix to this problem by designing a new fine-tuning method called LDIFS (short for $\ell_2$ distance in feature space) that, while learning new concepts related to the downstream task, allows a model to preserve its pre-trained knowledge as well. Through extensive experiments on 10 fine-tuning tasks we show that LDIFS significantly reduces concept forgetting. Additionally, we show that LDIFS is highly effective in performing continual fine-tuning on a sequence of tasks as well, in comparison with both fine-tuning as well as continual learning baselines.

Published in Transactions on Machine Learning Research (07/2024)
Fine-tuning can cripple your foundation model; preserving features may be the solution
Jishnu Mukhoti
jishnu@robots.o..cc.o
Deportment of Engineering Science, University of Oxford
Yarin Gal
yarin.galddes.or.ccccm
Department of Computer Science, University of Oxford
Philip H.S. Torr
philip.torrmeng.or.co uk
Deportment of Engineering Science, University of Oxford
Puneet K. Dokania
nunect@robots.oz.cccou
Department of Engineering Science, University of Oaford, Five
Reviewed on OpenReview: https: openreviee. net/ forun? id- kCh7
Abstract
Pre-trained foundation models, due to their enormous capacity and exposure to vast amounts of data during pre-training, are known have learned plenty of real-world concepts. An important step in making these pre-trained models effective on downstream tasks is to fine-tune them related datasets. While various finc-tuning methods have been devised and have been shown to be highly effective, we observe that fine-tuned model's ability to recognize concepts on tasks different from the downstream one is reduced significantly compared to its pre-trained counterpart. This is an undesirable effect of fine-tuning as substantial amount of resources was used to learn these pre-trained concepts in the first place. We call this phenomenon "concept. lorgetting" and via experiments show that most end-to-end fine-tuning approaches suffer heavily from this side effect. To this end, we propose a simple fix to this problem by designing new fine-tuning method called LDIFS (short for l distance in feature space) that, while learning concepts related to the downstream task, allows model to preserve its pre-trained knowledge as well. Through extensive experiments on 10 fine-tuning tasks we show that LDIFS significantly reduces concept forgetting. Additionally, we show that LDIFS is highly effective in performing continual finetuning on a sequence of tasks as well, in comparison with both fine-tuning as well as continual learning baselines.
1 Introduction
Foundation models like CLIP Radford et al. (0021) ALIGN Jia et al. (2021) and CoCa Yu et al. 2002 are trained using self-superviser met on hundreds of millions or even billions of samples scraped from the internet. This massive, compute intensive pre-training makes such models knowledge store on vast number of real-world concepts, enabling them easily transfer to wide variety of downst tasks and applications. Indeed, this ability to recognize real-world concepts and thereby transfer to downstream tasks is the primary advantage of such models and is the very reason behind their name Bommasani et al. 2021
While a foundation model can achieve impressive performance on downstream task often without even requiring a single training sample from the task itself Raddord et al. (2021), in order to maximise performance, it conventionally requires some form fine-tuning on the task at hand. There are multiple types of finc-tuning methods like lincar probing Radford al. (2021), prompt-tuning Zhou al. 2022b a), adapters Gao et al, (2021 Zhang et al. (202),, weight-space interpolation Wortsman et al. 2022b
Tharcoo al. (002aa) and full

---

Published in Transactions on Machine Learning Research (07/2024)
end-to-end fine-tuning Radford et al. (2021) Kumar et al. 2022 Goyal et al 2002 Xuhong et al. (018
Among these types, end-to-end fine-tuming is well-known to produce the best downstream performance.
It is worth noting that the pre-training dataset of a foundation model, owing to its massive scale, contains information about several thousands of real-world concepts. Hence, it is highly likely that the downstream dataset for fine-tuning the model will only contain a significantly smaller number of concepts compared to its pre-training set. A natural question that arises then is: How does end-to-en fone-tuning of a foundation model affect the vost knowledge it acquired through its pre-training? This is precisely what we aim to answer.
Through thorough study of popular end-to-end fine-tuning methods, we observe that for most of them, the fine-tuned model has significantly lost its ability to recognize real-world concepts outside the downstream lask, a phenomenon which we call concepl forgelling. This is highly undesirable as there are important use-cases where we would want the foundation model's vast knowledge on concepts to be preserved even after fino-tuning.
One important use-case, for instance, is the requirement for continuall fine-luning to update a pre-trained model with previously unknown knowledge. Several organizations have spent millions of dollars pre-training large foundation models from scratch Knight (2023) Even after such powerful pre-training, these models can exhibit gaps in their knowledge. Specific examples include ChatGPT OpenAl (0200) having a knowledge-cutoff date of September 2021 or CLIP Radford et al.
(2021) | failing to perform in niche downstream areas like satellite or medical imagery. Furthermore, fields like medicine are constantly updating with new information like new discases, new diagnoses and treatments etc. Therefore, having a fixed knowledge-bast base in medical
foundation models like MedCLIP Wang et al. (2022a) or MedPaLMM [Singhal et al. (2022) is not an option if we are to use them in practical downstream use cases. With their expensive pro-training process, ro-training a foundation model from scratch by combining new data with prior training data, is too expensive to be feasible. This requires the pre-trained foundation model to be fine-tuned on new data while preserving its prior knowledge, thereby necossitating investigation into fine-tuning methods which prevent concept forgetting.
Although conventional end-to-end fine-tuning methods generally suffer from concept forgetting, we find that. there can be a relatively simple fix this problem. In particular, if a fine-tuning method ensures that the fino-tuned model is close in some sense to the original foundation model, it can significantly reduce concept forgetting. One way to define the vicinity of the original model is in terms of distance in the parameter space. This is seen in the case of the L.2SP regularizer Xuhong et al. (2018). However, we find that it is much more effective to define vicinity in terms of distance in the model's feature space which captures its input-output behaviour. This leads us to propose a new regularizer, LDIFS (l2 distance in Feature Space), which minimizes the distance between the features of the original model and the model being fine-tuned during fine-tuning. Furthermore, we observe that simply preserving the last-layer features is not effective in reducing concept forgetting. Motivated from observations in ZZhng et al. (2018), we therefore preserve features extracted from different internal representations. We find that this relatively simple method of preserving the original model's 's features while fine-tuning on downstream task can significantly alleviate the problem of concept forgetting in the fine-tuned model, without affecting its performance on the downstream task. Empirically we show this through an extensive evaluation of LDIFS on 10 different downstream tasks.
Finally, since LDIFS preserves pre-trained knowledge during fine-tuning, as a natural extension, we study a continual setup with a sequence of fino-tuning tasks. In particular, our contimual sctup uses 3 sequences, each of 3 tasks. Again, in every evaluation setting, we find LDIFS to oulperform both fine-tuning as well classic continual learning methods in minimizing concept forgetting, without compromising performance on the finc-tuned tasks themselves. Thus, to summarize, our contributions in this work are as follows:
1. Investigate concept forgetting. To the best of our knowledge, we are the first to perform thorough analysis and evaluation of concept forgetting for fine-tuning multi-modal foundation models. We propose a simple way to quantify concept. forgetting and benchmark 6 existing end-to-end fine-tuning methods on 10 different downstream tasks to find that concept forgetting, as phenomenon, exists in all of them.
2. Analyze different end-to-end fine-tuning methods. We find a consistent ordering in concept forgetting for different fine-tuning methods. Particularly, we find the L2SP Xuhong et al. (0188) regularizer to outperform other fine-tuning baselines. We analyze why this is the case.
2

---

Published in Transactions on Machine Learning Research (07/2024)
3. Propose new regularizer for end-to-end fine-tuning. Analyzing L2SP helps us propose simple new regularizer (L.IIFS) which minimizes feature space distance between pre-trained and fine-tuned models during fine-tuming Minimizing feature space distance fine-tunes the model on the downstream task while preserving much of the input-output behaviour of the pre-trained model. This helps LDIFS outperform other existing fine-tuning methods in minimizing concept forgetting during fine-tuning.
4. A nudge towards continual fine-tuning. Finally, a natural extension of the evaluation setting we evaluate continual setup of 3 different sequences of 3 fine-tuning tasks each and find LDIFS to be superior to both fine-tuning methods as well as 5 classic continual learning basclines in preserving and accumulating knowledge in this setup.
2 A brief note on fine-tuning
Here we provide a description of CLIP Radford et al.] (2021) as our foundation model of choice and briefly discuss existing stale-of-the-art methods used for fine-tuning CLIP.
CLIP, a brief overview Broadly speaking, CLIP model has two components: i) the vision or image encoder fe. R.M.W. + RD, and ii) the text encoder fe, R + RD, CLIP is pre-trained on 400 million pairs of images and corresponding text. descriptions scraped from the internet. For pre-training, it uses a contrastive loss to maximize cosine similarity between the correct (image, text) pairs and minimize the same for the incorrect ones. Due to its large-scale self-supervised pre-training, CLIP exhibits impressive performance on several downstream tasks, often without requiring a single training sample from the task itself. However, in order to maximise performance on specific downstream task, the pre-trained CLIP model is conventionally fine-tuned further on the task itself. Below we provide brief description of such popular fine-tuning methods relevant to our study.
Zero-shot (ZS) In image classification, given an input image x and a set of K class names {c: as natural [
language text, the D-dimensional encoding for each class name (bc,)) = So, (c:) and the image d(x) - fa, (x) are first obtained. The text encodings abb (c:) are then used as parameters of a K-class linear classifier, and the classification inference on x is performed as arg max, (cx)))x(x). This is known as the zero-shot (ZS) prediction and CLIP's best model has been shown to have competitive ZS accuracics with fully supervised ResNet-101 on ImageNet Radford et al. 2202
Linear Probe (LP) In this case, an additional linear layer w E DxX is appended on top of the image encoder fa,, and the weights of this lincar layer are trained by solving a standard logistic regression problom (e.g., seikit-learn's 's LogisticRegresse initialized using the text representations (v)(c)) known as 7S initializatinn It trivial to note that in the absence of any training, LP boils down to ZS.
module PPedreoosa et al. (201T)). The linear layer w is normally
End-to-end fine-tuning While pre-trained CLIP encoder can obtain impressive ZS and LP accuracies several tasks, in order to maximize performance on specific downstrcam task, the general rule of thumb is to initialize a model from the weights of the pre-trained encoder and then fine-tune the model end-to-end on the downstream task. Here we list some of the most popular end-to-end fine-tuning methods which we study in this work. We provide a more detailed discussion on the different types of fine-tuning methods $
1. ZS-init- CE Radford et al. (2021) :: This is the classic end-to-end fino-tuning method where, similar to the LP,, 7S initialized linear head w IRD + R is appended to the image encoder Ja,- However, differently from the LP, parameters of the entire model 0 = {0,, w) (including the image encoder parameters) are fine-tuned using a cross-entropy loss LCE.
2. LP-init-CE (LP-FT) Kurnar et al. (2022): This is similar to ZS-init-CE but instead of initializing the appended linear head via ZS, is initialized by performing linear probing on the downstream task first Once the linear head is initialized, the entire model is end-to-end fine-tuned using LCg-
3. ZS-init-L2SP Xuhong et al. 2018): : In addition to the cross-entropy loss LCE, this method uses additional regularizer to minimize the lg distance between the pre-trained and fine-tuned image encoder
Note, the vision encoder first extracts D. dimeusional image features and then projects them to D dimensional space via linear embedder w. : RP + RD. Similarly for the text. encoder.
Thh natural language class names are often augmented with prompt like Fan image of (class name)."
3

---

Published in Transactions on Machine Learning Research (07 (07/2024)
weights, thereby trying to keep the fine-tuned model weights close the pre-trained ones. Let the pre-trained image encoder weights be Oo(o) and the encoder weights at time step during fine-tuning be (e(t)- Then, the fine-tuning loss in this case becomes
LLSP LCE + A1.spl100css Occoy
Note that the l2 distance is only computed between the weights of the pre-trained and fine-tuned image encoders.
4. UP-init-L2SP. This is similar to ZS-init-L2SP but the linear head w is initialized by performing linear probing on the downstream dataset first. The loss for end-ta-end fine-tuning then is the same as in Equation ①
5. FLYP Goyal et al. 2022): The Fine-tune Like You Pre-train or FLYP bascline finc-tunes both the image and the text encoders of CLIP and uses contrastive loss Lennt instead of cross-entropy LCE for fine-tuning on the downstream task. The parameters being fine-tuned here are = {0,, 8t), i.e., both image and text. encoders CLIP.
6. FLYP-CE Goyal et al. 2022): This is an ablation on FLYP where, instead of using contrastive loss, the fine-tuning is done using cross-entropy loss LCE, taking the cosine similarities between image and text embeddings as logits, Note that similar to FLYP, in this case as well, both image and text encoders are fine-tuned end-to-cnd.
3 The crippling effect of end-to-end fine-tuning
The contrastive pre-training dataset of CLIP contains 400 million (image, text) pairs scraped from the internet Radford et al.] [202T). Consequently, any downstream task that CL.IP is fine-tuned on is highly likely to contain only a small fraction of concepts compared to what it has already been exposed to during pre-training. To investigate the impact of fine-tuning, here we perform a thorough study benchmarking 6 fine-tuming methods on downstteam classification lasks. We find that for most of these methods, while the finc-tuned model attains excellent improved performance on the downstream task itself, its general ability to recognize concepts outside the task is significantly reduced over the course of fine-tuning. We call this phenomenon concept forgetting and find this to be undesirable effect of most fine-tuning methods. To explore this in detail, in this section, we first discuss how we quantify concept forgetting, then we propose our benchmarking setup for fine-tuning methods and finally present our observations.
3.1 Quantifying Concept Forgetting
During 7S and LP evaluation of a model (refer $2) the pre-trained image encoder weights O, remain frozen and unchanged irrespective of the downstream task at hand. Therefore, ZS and LP performance specific downstream task can be good indicator of the pre-trained model's existing knowledge about the task. This is the hypothesis we base our analysis on.
While ZS accuracy based on how well the text encoder representations can form a linear classifier in the image encoder's s feature space, LP performance is indicative of whether the image encoder representations are linearly separable in the first place. Note that this distinction is important. Fine-tuning the weights of just the image encoder 0g can lead to situation where its representations are no longer well-aligned with the text. encoder. Even so, for a given task, if the image encoder representations are linearly separable, captured by its LP accuracy, it shows that the model is still able to recognize concepts involved in the downstream task, thereby indicating the preservation of knowledge on the task. Thus, after fine-tuning, ZS accuracy may not reflect a model's ability to recognize concepts in a task and LP accuracy is a better candidate to do so. Wo illustrate this point Figure 0
Therefore, in order to quantify concept forgetting on particular task defined on a dataset D, we measure the difference in LP accuracy between the pre-trained and the fino-tuned image encoders on D. To formalize
STo the best of our knowledge, I.P-init-T.2SP, has been evaluated henchmarked prior this work.
4

---

Published in Transactions on Machine Learning Research (07/2024)
this, let feweo be the pre-trained image encoder and fa, the one obtained via fine-tuning dataset Dn- Furthermore, let ALr(Jo, D) represent the LP accuracy of image encoder So, on dataset 'D. Then, we define the change in LP accuracy on D between pre-trained and fine-tuned models ALP(D, fo., fem.oo) (or in short, ALP) as:
ALr(D, So.: Jowan) = Air(Jo. D) - ALL(SO000) D)
(2)
Note, fo, is finc-tuned on Dr, not on D. The dataset DD here represents a target dataset on which we would like to conntor/quantinn con- cept forgetting. Clearly, if we use Dtt instead, we expect LLP f fo. to increase over the course of fine-Luning However, the main objective here is to quantify the effect of fine- tuning on task defined on dataset D that was not part of the fine-tuning procedure itself (D DD.. A negative value of ÅLP indicates concept forgetting, a zero indicates knowl- edge accumulation and a positive value in- dicates knowledge gain or positive forward transfer Lopez-Paz & Ranzato (0117) on the lask under inspection. We would like to high- light that the dataset DD here is a user-defined dataset on which monitoring the effect of fine- tuning would be desirable.
LP
LP
25
Fine-tuning
Figure Pictorial representation of classifiers from zero-shot (ZS) and linear probe (LP). While ZS separate representations pre fine-tuning, the representations learnt after fine-tuning may not necessarily be lincarly sep- arable by the same ZS classifier anymore. However, if an LP is able to classify representations after fine-tuning, the model can still recognize the concepts involved.
Catastrophic forgetting vs concept for- getting Catastrophic forgetting McCloskey Cohen (1989): Kemker et al. (0018): Kirkpatrick et al. 2017 is a well-known phenomenon which signifies how, when a model trained on a new task, its performance on the previous task drops catastrophically. For example, if a model trained on ImageNet is then trained on say CIFAR100, it significantly loses ils performance on ImageNet. Though concepl sorgelling mentioned in this work is very similar to catastrophic forgetting and do not claim much conceptual novelty here, there is subtle difference that we believe requires distinction between the two. The major difference lics in the fact that in the pre-foundation model era, the pre-training datasets were much smaller and the pre-training tasks were fully supervised, for example, classification on the ImageNet dataset. Therefore, it was possible to roughly quantify the degree of damage the model had on the pre-trained task (using trained dataset) once it was fine-tuned on new downstream task. However, in the case of foundation models, it is not trivial to quantify what the model knows as it was pre-trained several millions or even billions of examples using self-supervised training, and the pre-training dataset is also often inaccessible. Therefore, it is not possible to quantify exactly what the fine-tuned model forgot and the catastrophe therein. Hence, it is necessary to devise poking and probing mechanisms, similar to the ones we present in this work using LP, to quantify the effect of fine-tuning on a smaller but relevant domain of concepts (represented by the dataset D above).
3.2 Benchmarking concept forgetting
To quantify concept forgetting, here we use CLIP Radford et al. (2021) ViT-B/32 pre-trained on the OpenAl dataset and released in the OpenCLIP repository Ilaarco et al. 2021 and measure its LP performance over fine-tuning on 10 different image classification downstream tasks with a high variability in their semantic concepts. These datascts, along with their respective train/test splits are:
1. Stanford Cars Krausc et al. 2013 containing 16, 185 images of 196 classes cars with a train/tost split of 8144 and 8041 images respectively,
2. CIFAR- 1ARR-101110 10/100 (C10/100) Krizhevsky et al. (009 containing 60, 000 images of vehicles, flora and fauna, divided into 10100 classes with the train split having 50, 000 and 10, 000 images respectively,
5

---

Published in Transactions on Machine Learning Research (07 (07/2024)
25-INB-CE
POP
FUP-CE
-23
-100
AvA
Tannng complesinn
Trariig comp edon
13 complesion
Pariing corpiction
Tatting completion
Figure 2: Test set ALP for models fine-tuned on EuroSAT using different fine-tuning methods. While EuroSAT ÅLP rises, ALP on all other datasets is almost always negative throughout the fine-tuning with a sole exception of DTD when fine-tuned using LP-init-L.2PP See $32 for details.
CIFARIO
CIFARIOO
2.-
-5.0
A
-225
1.0
03
20 60 60 06 103
50
60 203
27
80 300
23
46
55
96.
complation
completion
(a)
(b)
Figure 3: Test set ALP evaluated on different datasets for models fine-tuned on EuroSAT via various fine-tuning methods. The L2SP baselines (particularly LP-init-L2SP) have the lowest negative ALP datasets other than EuroSAT.
3. DTD Cimpoi et al. (0144) containing 3760 images of 47 classes of textures found in the wild with 1880 images each in the train and test sets,
4. Eu Helber Helber et al. 2019 containing 25, 000 samples with 10 categories of satellite images of landscapes and 19, 600/5400 training/ test images respectively,
5. GTSRB Stallkamp al. (2012) containing 39, 270 images of 43 classes of German traffic signs with 26, 640
training images and 12, 630 test images,
6. MNIST LeCun et al. 1998 containing 60, 000 training images and 10, 000 test images of 10 handwritten digits from 0 to 9 in grayscale,
7. RESISC45 (R45) Cheng et al. 2017 containing 25, 200 samples with 45 classes various remote sensing
image scenes with the train/test split having 18, 900 and 6300 images respectively,
8. SVHN Netzer et al. (2011) containing total of 99, 289 colour images of street view house numbers, each image being calegorized into one of 10 digits with 73, 257 training samples and 26, 032 test samples.
9. Image.ee Deng et al. 2009 containing a total of 1.28 million training images and 50, 000 validation images of 1000 classes.
Finally, for this study, we use the 6 end-to-end fine-tuning methods discussed in 2 The training details can be found in Appendix A
Fine-tuning causes concept forgetting. In Figure 2 we present the ALP for models fine-tuned EuroSAT using the 6 fine-tuning methods discussed in 2 Across all fine-tuning methods, we observe that while as expected, the performance on EuroSAT test increases and ALP is positive (also see Pigure Bel, performance on 8 other dounstream datasets decreases and ÅLP for all of them is negative. This indicates that all fine-buning methods suffer from concept forgelting. The only exception to this when we evaluate on DTD and the model is finc-tuned using LP-init-T.2SP. We observe ALP to be mostly positive before it goes to zero at the end of fine-tuning. This is an interesting case as it indicates that LP-init-L22PP on EuroSAT might actually be helping increase knowledge about DTD before the fine-tuning becomes too specific to ET. This may be an example of positive forward transfer Lopez-Paz & Ranzato (2017): : Chaudhry et al.
(018) and exploring why such knowledge accumulation happens is an interesting avenue for further research.
Next, we compare between these fine-tuning methods by showing ALP for different downstream datasets in Figure 3. where it is evident that LP-init-LL2 L2SP consistently outperforms other baselines lowering concept foryetting and preserving the fine-tuned model's original performance across multiple dounstream tasks. This is observable from its low negative ALP compared to other baselines. While it suffers initial dip in
6

---

Published in Transactions on Machine Learning Research (07 (07/2024) /224)
110mm Buall EuroSAT
1|0.g Buoil GTSRB
20
175
TLPP
-
01
FLYP-DE
Z5-init
0100
ZS-initL2SP
LRinit-CE
LP-ini-L2SP
0.0
20
16 Training complction
% Training completion
% training completion
Figure 4: lp distance in the parameter space (/0oct) - co( [1 of the image encoder over the course of fino-tuning. Except L2SP, all other basclines diverge away from their pro-trained counterparts.
performance in the early stages of fine-tuning, in the later stages, LP-init-L2SP regains the accuracy, often ending up with a near zero ALP. Its impressive performance on concept forgetting however, does seem to come at the cost of a relatively lower ALP on the fine-tuning task, i.c., EuroSAT, itsclf. In this regard, note that most fine-tuning methods are specifically designed to maximise performance on the downstream fine-tuning task. Thus, the performance in Figure 3e is conventionally the primary evaluation criterion for fine-tuning method. However, shown here, concept forgetting is undesirable additional effect of fine-tuning and requires its own evaluation. Our observations for other datasets are similar (sec Appendix C In what follows, we first investigale L.P-init-.22PP further lo gain insights on why it preserves concepts better than other baselines, and then use those insights to propose a new fine-tuning method that significantly outperforms all other basclincs.
4 Can preserving features help?
The T,2SP regularizer (eq. ( enforces the model fe at time-step of fine-tuning to be the vicinity of the pre-trained model fe..o: by minimizing the Eg distance between the two the paramcter space. As evident from $ this simple regularizer has a promising impact on the model's ability to avoid concept forgelting. To understand how correlated the parameter space l distance [|0u()) ou is to concept forgetting, in Figure we show how & distance changes as we fine-tune different datascts (EuroSAT, GTSRB and SVHN) using all the discussed fine-tuning methods. From Figure 4 relatively speaking, all the fine-tuning methods except the two L2SP baselines (ZS-init-LL25P and AP-imit-L.2SP) cause the parometers of the fine-tuned model to move away from its pre-truined counterpert.
Though regularizing the fine-tuned model to be in the vicinity of the pre-trained one shows a promising effect in preserving concepts, vicinity in the parameter space need not necessarily capture the input-output. behaviour of the pre-trained model. In fact, it is trivial to construct two different sets of model parameters, far in l distance, outputting similar values in a specific domain. Additionally, regularizing in the parameter space might not keep the fine-tuned model in the desired vicinity that preserves the pre-trained knowledge and performs well on the downstream task at the same time. Therefore, we conjecture that the vicinal distance in feature space as opposed to parameter space, might be a belter indicator of the similarity of encoded concepts in the model. Indeed, in the end, the internal representation space where models encode patterns. Thus, in Figure 5 inspect lg distance in the feature space over fine-tuning using the following distance function:
N D Da (xi) - Po (x
((Ov(0):O(()) D)
(3)
N
where, Po.,, (x;) represents the features of the model with parameters ou(1) at time for a given sample Xs, and N is the number of samples in the dataset. Note that the feature vector &o (x:) is obtained by concatenating various internal representations (not just the last layer features) of the network architecture, similar to the perceptual features presented in Zhang et al. (2018). The exact details as to how we compute the concatenated feature vector for a ViT-B/32 B/22 model is mentioned in Appendix B and similar plots for other datasets is shown in Appendix
A strong correlation between concept forgetting and featuro-space distance. Similar our observations in Figure (parameter space), we note that except L2SP, all other fine-tuning methods suffering significantly from concept forgetting cause the fine-tuned model to move away from the pre-trained model
7

---

Published in Transactions on Machine Learning Research (07 (07/2024)
89 694223
0071
0. 6025
88050
88033
16015
8.03
2 21000
FUP CE
FUP
20.010
20 6813
ges000
1000
geass
caas
100000
g8000
6025
20.023
St.tt
E 0000
- 0666
6.000
15 Training completion
95 Training compiction
Traninng compictin
% Training completio
Trainin completion
Figure 5: &2 distance in feature space d(O Peco), D) (see eq. (3) between image encoders computed
over the course of fine-tuning on EuroSAT. The & distance is computed for EuroSAT train and test sets as well as CIFARIO, CIFAR100 and GTSRB datasets.
dg EUFOSAT
G !I! GTSRB
20.0
12.
15.0
FLYP-CE
25 nit CE
200.0
25-nit-125P
13
LPInIE
110mg
18mm
2.3
LNinicL25P
0.0
25-nIt-LDIPS
29
LP LDIFS
% Training completion
% Training completion
Training completion
Figure 6: la distance in weight space [|0oce) Ovto) time-step fo..; and the pre-trained image encoder fe.co) over the course of fine-tuning.
between the image encoder fine-tuned the current.
in terms of feature-space distance as well (Equation 3)- In case of 7S-init-L.2SP and LP-init-L..SP, while initially diverging, the tune models recover their pre-trained behaviour to certain extent in the later stages of fine-tuning. It is important to note that this observation is consistent for the fine-tuning training and test sets as well as for all other datasets.
Based on the results shown in Figure 3 and Figure we conclude that the fine-tuning methods which cause the model significontly diverge away from the pre-trained model either in parameter space or in feature space suffer more from concept forgetting. This observation, combined with our conjecture above leads to natural extension of fine-tuning where we use d((er()): ((<o)) D) (distance in the fcature space) the regularizer. We call this regularizer LDIFS: & distance in Feature Space. The complete fine-tuning objective then becomes:
LLDIFS = LCe + ALDIES d(f.(r): Qu(0). Deraiu)
(4)
where Dusin is the training or fine-tuning set and ALDIFS is the regularization coefficient. Note that similar to 1,2SP, we can initialize the linear head with both zero-shot weights linear probe weights leading to two variants: ZS- IS-init-LDIFS and LP-in
Analysing LDIFS on parameter space and feature space distance: In Figure 6 and Figure we plot the & distance in trrameter/weighh and feature space respectively (same as Figure and Figure but with the LDIFS added in. In the weight space, LDIFS, while lower than other finc-tuning methods, has a relatively higher &2 distance from the pre-trained model compared to L2SP. On the contrary, in the feature space, LDIFS consistently gets the lowest distance from the pre-trained model. This expected and was indeed the purpose behind LDIFS's design which is to minimize the difference in input-output behaviour, captured through feature space distance between the fine-tuned and pre-trained models and not necessarily constrain the parameter space of the fine-tuned model to lie close to the pre-trained model. Finally, note from Figure 7 that although we are applying the LDIFS regularizer only on the EuroSAT AT samples during fine-tuning, the preservation of features extends even to other datasets like CIFAR1O, CIFAR100 and GTSRB, as seen from the low feature space distance even on these datasets. This indicates, that we don't need third party datasots during fine-tuning to preserve the pre-trained modcl's behaviour and regularizing on samples just. from the fine-tuning dataset can be sufficient.
LP-init-LDIF' significantly reduces concept forgetting. First, we evaluate LDIFS on the same setting as in $ and present the ALP over finc-tuning in Figure g (additional results presented in Appendix C Second, in Table [ we report the Arp% on the fine-tuning test set and the ALP averaged over other tasks. ALP % measures performance on the fine-tuning task itself and ALP provides an estimate of the fine-tuned model's level of concopt forgetting on the remaining tasks. For downstream tasks which are not ImageNet, we report ALP averaged over the remaining tasks, except ImageNet. For ImageNet, we leave out CIFAR-10/100 when calculating performance over other tasks since all of CIFAR-10/100 1000's classes within the set. of
8

---

Published in Transactions on Machine Learning Research (07/2024)
E
206
85use
6.17
FSP
022
30000
8033
25
rePer
20.0055 6025
2000
6o 655
50
gesess
4/1.004
(PA--GE
0025
g2.002
000000
Sccce
5.00
0.1.003
45
86
Tridning comppetion
335
23
Taaninn completion
50 135
Training completion
20 to Mi
60
LPANE-LDIT
Figure 7: l distance in feature space d((wce); Po), D), between image encoders, Seacs and Se computed over different finc-tuning methods for models finc-tuned on EuroSAT.
CIFARLDO
GTSRB
15
EuroSAT
8.6
-25
2.-
2.0
-12
-125
-15.0
-15
55 Training completion
Training completion
Traning completion
5 Trininn completion
35 Tranning completion
Figure 8: Test set ALP evaluated on different datasets for models fine-tuned on EuroSAT. LDIFS (Our) baselines (both 7S-init and LP-init) provides the best results in terms of avoiding concept forgetting without affecting downstream performance on EuroSAT.
Dataset
Fine-tuning baselinas
Aspff)
Aur(t)
Aur(1:
Asr(")
AAr(f)
ALp(11
Cars
56.00 06
83.48
-1.36
$1.95
-0.63
$2.64
58
0.47
-0.36
85:26
-0.15
CIFARIO
97.11
-3.35
177.65
-1.37
97.33
-1.0
97.71
-0.81
97.7
1.04
97.05
1.18
97.30
1.08
97.24
1.18
DIFARTO
58.05
-1.16
68:77
-3.5
85.6
-0.96
85.11
-6.11
$7.81
0.63
66.01
1.43
87:84
0.83
85:50
0.86
76.12
73.00
-3.41
7T.18
72.18
-1.76
73.88
-3.71
74.01
0.01
28.14
11
73.97
0.53
EusBAT
2.55
MKNIT
96.02
58.63
7.53
89
-2.03
98.18
1.49
3.8
89.30 02-32
2.64
RESISC4S
55.84
5.42
50.70
3.52
05.76
3.70
05.30
2.27
94.00
0.01
$4.13
0.06
00.22
0.41
05.13
0.0
SVIN
07-48
-10.24
97-4
-10.00
07.3
-11.12
90.3
-8.73
99.5
-2.78
96.54
-2.31
96-97
-00.08
96.35
-0.29
EmageNNet
50.26
-1.6
22.15
-1.20
82.00
-1.25
-0.87
-0.21
88.75
-03.
0.33
82.31
Table 1: Test set accuracy A1p% and average ALP computed over other datasets for models
fine-tuned on 10 image classification tasks. ALP shows performance the fine-tuning task itself and ALP shows the level of concept forgetting on other tasks (higher ALP shows lower concept. forgetting and vice versa). The best ALP numbers are shown in bold.
ImageNet classes. We use the remaining 7 tasks to quantify forgetting. Full set of results for all baselines can found Table and an alternative visualization of these results are in Figure I4 of the Appendix. Our observations are as follows.
1. Competitine dounstream accuracy: From Table [ we note that there is no clear winner in terms of accuracy on the downstream fine-tuning task itsolf. Additionally, from Figure and Table L it is also evident that LDIFS is competitive with other fine-tuning baselines in terms of accuracy the downstream fine-tuning task itself.
2. Minimal concept forgetting: LP-init- LD-Fn, significantly minimizes concept forgetting over the course of fine-tuning. This is evident from Figure which shows its noticeably higher ALP on other downstream tasks over the course of fine-tuning. It is also apparent from its consistently high ALP scores in Table i Particulatly, LP-inil-LDIF'S gets the highest average AL. on other tasks for 8 out of 10 fine-tuning cases. This indicates significantly lower level of concept forgetting.
3. Positive Forward Transfer: From Table I also observe that in 8 out of 10 fine-tuning tasks, LP-init-LDIF'S results in positive average ALP other tasks, thereby generally achieving a positive forward transfer. This is not the case for other basclincs where ALP is generally negative indicating concept lorgetting. The two exceptions for LP-init-LDIFS are when the finetuning tasks Dn. are Stanford Cars and SVHN, in which case the average ALP on other tasks is negative. This observation also highlights how the ordering of fine-tuning tasks can also impact forward transfer. For instance, from Table g when fine-tuning on EuroSAT, LP-init-LDIFS achieves ALP of +3.03 on SVHN, whereas in the reverse scenario, fine-tune finettuning on SVHN leads to negative ALP of -0.78 on EuroSAT.
9

---

Published in Transactions on Machine Learning Research (07/202)
FLVP
PINP.CR
25-inib-CR
L.Pinli-CR
28-1nit-1.255
L.Piiit-125P
25-Init-LDIIS
L.--iSI-T.DIPS
Aurth)
Ase(T)
Aue(1)
Aur(t)
Aur11)
ALPtT)
ALP(7]
AAu(t)
Aur(t)
(Aurt))
Nur(1)
[ALPET)
CRAP
VBL6
Vis12
12:36
65.0
-3.24
-9.7
82:38
58.38
-2.46
-2.16
62
26.2:
-300
-2.0
K2.32
63.36
-2.37
-1.78
45.11
10.9
-1.13
-1.00
50.75
59:10
-0.45
-0.24
82.34
85:00
0.02
0.07
62:21
66.81
0.13
0.3
V7.4
59.01
60:00
60.91
-1.59
86.91
-6.87
50.87
-0.45
67.74
0.12
57.85
PLAVA
V816
61.79
61.54
31.18
61.36
-3.04
79.07
-1.60
50.13
-11
01:21
81
0.01
DENO-2
VBL4
V114
85.07
-1.92
85.18
67.
-1.46
81.16
85.61
-0.04
-0.00
07.02
84.5
-0.66
-0.19
88.00
07.04
0.0
0.05
50
0.08
MAP
VLI6
25.86
4.26
90.04
-8.00
35.00
-2.04
86.1
-1.82
85.47
85.0
0..1
Table 2: Test set accuracy ALP % and average ALP computed for different architectures of CLIP Radford
et al. (021), FLAVA Singh el al. (2022), DINOv2 DINOv2[Qqual Oquab et al. (2023) and MAE He el. al. (2022) all fine-tuned on ImageNet. The best ÅLP numbers are shown in bold.
We have similar conclusions on 3 other CLIP architecturos: ResNet-50, VIT-B/16 and VIT-L/14, FLAVA ViT-B/It 16 Singh et al. 2022), a DINO-v2 ViT-B/14 and ViT-L/14 Oquab et al. (2023) and Masked Auto- encoder He et al. (2022) VIT-B/16 and ViT-I./16 models. We present the details and results next.
Generality of concept forgetting and LDIFS beyond CLIP VIT-B To study the relevance of concept forgetting and LDIFS beyond a CLIP B/3- model, we experiment with other architectures and models. Among text-image multi-modal foundation models, we use CLIP and FLAVA Singh et al.] (2022). For CLIP, we add the ResNet-50 (RN50), VIT-B/16 (VB16) and ViT-L/14 (VL14) architectures in addition to ViT-B/32. For FLAVA, use the ViT-B/10 /6 architectue. In addition, vision only foundation and pre-trained models well, in particular DINOv2 Oquab et al. (023) and Masked Auto-Encoders (MAE) He et al. (2022). For DINOv2, we use ViT-B/14 B14 and ViT-L/14 architectures and for MAE, we use ViT-B/16 and ViT-L//6 architectures. All the models were fine-tuned on ImageNet. using the same training recipe as mentioned in Appendix A To quantify concept forgetting, we use DTD, EuroSAT, GTSRB, RESISC45 and SVHN as other datasets and report ÂLP avcraged over those datascts. Note that the FLYP and FLYP-CE baselines particularly work with multi-modal foundation models, so we don't evaluate these two baselines for DINOv2 and MAF. Finally, 7S-init fine-tuning on DINOv2 and MAE are done using randomly initialized linear head. We present results from this experiment in Table 2 Following are our observations:
add experiments
1. Concept forgetting happens across architectures and foundation models. From Table 2 we can see a consistently negative ALP indicating concept forgetting for all models and architectures and all classic fine-tuning baselines (except our proposed LDIFS baseline).
2. Larger models forget less. For cach model category, i.c., CLIP, FLAVA, DINOv2 and MAE, we obscrve progressively lower concept forgetting as model size gets larger. The ViT-L models consistently forget. less than their VYT-B counterparts.
3. CLIP and DINOv2 are competitve and outperform FLAVA and MAE. However, it is not clear if this is an artefact of larger pre-training sets for CLIP (WTT-400M) and DINOv2 (IVD-142M) as opposed to FLAVA (PMD-77M) and MAE (ImageNet-1K 1.3M), their pre- preiingg self-supervised loss functions some combination of other factors. This is interesting to investigate for future work.
4. LP-init-LDIFS is the consistent winner. For each model and each architecture, we find our proposed JP-init-LDIFSS to provide the highest Aur values which are mostly positive, indicating both minimized concept forgetting as well as positive forward transfer in many cases.
5 A nudge towards continual fine-tuning
Our results above indicate that fine-tuning on LP-init-DDFFS can make the foundation model learn new downstream information without forgetting pre-trained concepts. A natural question that then arises is: Can we fine-tune on a sequence of dounstream tasks without foryetting concepts? For an ideal fine-tuning method, the final model should attain state-of-tho-art performance on all fine-tuned tasks while still maintaining its pre-trained knowledge. We empirically investigate this question by training on 3 sequences of 3 tasks each: a) SVHN + C10 - R45, b) SVHN - C100 - R45 and e) SVHN -> Cars R45. Note that this setup
10

---

Published in Transactions on Machine Learning Research (07/2024)
2011
641
Figure 9: ALP for sequence: SVHN + CIFAR10 + RESISC45 (left plots) and ii) SVHN + CIFAR100 RESISC45 (right plots). Vertical red line indicates a switch in the fine-tuning tasks.
is similar to continual learning Chaudhry et al. (2018) Rebuffi et al. 2017 Lopez-Paz & Ranzato 2017
but for pre-trained foundation models. Due to their impressive performance on wide range of rownstreemm tasks, continual fine-tuning of foundation models is relatively unexplored in the literature. Nonetheless, as stated in I this is an important problem to investigate from the perspective of updating a foundation model with new previonsly unknown knowledge without forgetting previously known ones.
Quantifying concept forgetting in continual sctups. Let the sequence of finc-tuning datasets be DI - - ..., Da and the corresponding sequence of models be fuo...., Sor with Sog being the pre-trained model. In order to then quantify concept forgetting task D over this sequence of models, extend the ALP from Equation
2 as follows:
(D, Sou: Um So. .o.. Sox ..) - Ar.(Jon D)
max ie (0(.......)))
ALP(Jo; D).
(5)
Hence, similar to Chaudhry et al. (0118), we find the difference in LP performance between the final fine-tuned model fo, and the model having the maximum LP performance in the sequence {fau, ffa,.fe..... fe. } This accounts for the possibility of positive forward transfer on D over the sequence of fine-tuning tasks.
LP-init-LDIFS significantly reduces concept forgetting in continual setups. In Table 3 we present the ALP and ALP for the continual setup for all fine-tuning baselines. For each task sequence, we report the performance datasots in the sequence in the first three rows and in the fourth row, report the average performance on other datasets. From Table 3 we observe:
1. For all 3 sequences, L.-init-LDIIS has minimal concept forgetting on tasks which appear earlier in the fine-tuning sequence, as well as other datasets which not used for fine-tuning.
2. At the same time, LP-inil-LDIIS is very competitive on accuracy with the best fine-tuning methods and consistently outperforms L2SP on accuracy on the last fine-tuning task, i.e., RESISC45.
These observations are further complemented in Figure 9 where we can see I.P-initLLDIFS to have minimal forgetting on prior tasks, SVHN and C10/100 without compromising on performance on the last task.
LP-init-LDIFS outperforms classic continual learning methods. Due the similarity of the proposed setup in this section with classic continual learning, we empirically compare performance of L-P-init-LDIF with 5 well-known continual learning baselines: LwF Li & Hoiem (2017). LFL Jung et al. (016)). iCARL Rebulfi et al.] (2017), Distillation + Retrospection (D+R) Hou et al. (2018) and ZSCL [Zheng el al. 20223
on our sequence setup. Results are in Table | Again, the results indicate that LP-init-LDiFs performs better than all other continual learning basclines, both preventing forgetting as well as obtaining the best performance on the last task. This is evident from its consistently high ALP and ALP on all 3 sequences. Finally, in Appendix C.7 we provide additional results on ablation comparing T.P-init-I.DIFS with joint.
training Li & Hoiem (2017) on our 3-task setting. In addition, we provide comparison of LP-init-LDIF' with continual learning methods Wang et al. (002cc bb: Smith et al. (2023); Thengane et al. (2022) Zhang
et al. 2023 which utilize pre-trained encoders to improve performance. We perform this comparison on Split.
ImageNet-Re Wang et al. (2022b), a well-known clas--incremental learning benchmark. In these experiments, LDIFS outperforms baselines which use pre-training and performs competitively with joint training.
6 Related Work & Discussion
In this section, we first provide a brief discussion on the different types of fine-tuning approaches apart from end-to-end fine-tuning which have been applied on foundation models. We also discuss the relation and differences between our proposed LDIFS regularizer and previous works on continual learning which particularly featureddistllattion methods.
11

---

Published in Transactions on Machine Learning Research (07/202)
Pinestuning botelison
Fine-tune
Eval
FLYP
TLYP-CE
25-Inli-CE
LP-inh-CE
25-1alt-1a5p
LP-Suh---2p
25Iat--DIPP
Aur(h)
Auriti
AAr(tt)
Aurlf!
Aur(1)
Aurl))
Aur(1)
Aur(")
Acrlt1
(uritt)
Aurlt:
AcriT1
BVR
-T:
50.55
-6.77
11.83
-T-13
20.22
-4.45
10.30
-1.41
91.01
-4.55
91:13
-1.31
18.05
-U.41
US.
-3.16
01.61
-1.00
-2.31
$5.85
-1.57
96:31
-1.28
-0.33
07:36
-0.35
07:16
-0.21
92.36
05:01
05.0
Otlers
-6.0
73.91
-4.0
61.2
-0.05
50.91
-1.24
82.13
-1.82
59.27
-0.01
09.49
-0.3
06.30
0.1
07 05
SVRN Hp C100 R45
CIFAR1N
4.38
-8.00
90.04
79:22
-0.35
51.85
95.21
-7.18
6.15
53.08
93.4
-3.44
281
12.61
08.81
-3.68
2.79
14.13
16.2
-1156
199
85.72
93.21
-1.93
4.29
58.50 05
03.89
-0.5
3,65
55
26.11
Dibers
-5.23
63.05
-6.05
93.07
-405
$5.70
86.14
-1.61
87.52
-0.37
00.04
55.36
-0.25
B0:12
SVRN
-1.26
19.07
-1.01
09.76
-1.4
15.08
-0.75
19.06
-1:36
16.42
11.64
13.05
0131
08.36
0.17
05.0
SVIN 345
4:10
96 46.40
3.41
95.06
3.60
95.17
3.0
66.35
0.83
00.13
0.33
01.11
3.94
Other
-0.00
01.00
-5.48
63.2
-4.0
$3.36
-4.31
$4.30
-2.01
65.24
-1.67
07.15
-53
05.75
0.23
80.00
Table 3: ALP and ALP % for models tinettned on (SVHN, C10, R45), (SVHN, C100, R45) & (SVHN, Cars, R45) sequences. The first 3 rows show performance on finettned tasks and the third row shows averaged performance on 6 other datasets.
Learning
Fine-tune
Icval
Lwo
LPL
Dell
Li-tmtt-LLLLs (Uurrs)
Aupif)
Aupit)
Aurit)
Aurit)
Aurit)
Air(1)
Agalf)
Aur(t)
ALrCh
SVHN C5
90-15
-3.21
$1.5
-3.67
91 02
-2.78
91-3
-3.35
M2.7
-0-41
96.
SVHN
94.22
2.58
$3.9
2.83
95.72
3,63
$4.94
3.52
94.89
M5.0
Others
-4.2
80.75
-3.76
$1:31
-1.11
80.75
-39
61.00
-2.8
83.1
0.1
87.08
SVILN
4.54
80.05
-9.08
00.20
-4.33
90.37
-3.23
92.3
-3.92
91 91.81
-0.85
06.32
CIEABLI
-1.23
83.25
-3.01
53.00
-3.13
84.08
-26
54.82
-2.15
85.07
-0.3
80.54
Others
-4.13
83.75
-3.8
12.04
-1.02
83.62
-3.43
12.17
-3.11
62.86
-0.25
H9:12
SVII
-1.61
91-43
-2.90
07.74
-3.13
91.75
-5.44
80.06
-277
92.95
-0.17
86.3
SVIN Clars 8655
Care
-2.79
81:69
-2.64
3.55
-28
81.7
-3.13
02:11
-1.81
02:08
0.47
#4:23
Orhers
-3.89
81.85
-2.8
83.1
Table 4: Aur and ALp% comparing LDIFS with 5 classic continual learning methods on the 3-task setup.
Fine-tuning in foundation models: There are many flavours of fine-tuning which improve on CLIP's ss zero-shot performance. A popular approach is prompt tuning where, methods like CoOp Zhou et al. 2022b)
CoCoOp Zhou et al. (2022a), TPT Shu et al. (2022), Chain of Thought prompt tuning Ge et al. (2023)
instcad of hand-crafting prompts, learn the prompts specific to the finc-tuning task. Another category of
methods uses adapters: CLIP-Adapter Gao et al. (2021), , Tip-Adapter Zhang et al. (202T), Prompt Adapler Sun et al. (2023), SVT-Adapter Pantazis et al. (2022), which works on the principle of combining pre-trained features with a small non-linear adapter network where the adapter is fine-tuned on the downstream task at hand. Along with end-to-end fine-tuning, there are weight space interpolation methods like Wise-FT Wortsman et al. (202bb), PAINT Thharco et al. (202bb), task arithmetic Maarro et al. (2022a) and model soups Wortsman et al. (2022a) which look at interpolating between pro-trained and fine-tuned models in the weight space to achieve the best of both worlds in terms of reamst.eem task performance and robust.nsss to distribution shift.
In Table g of the appendix, we compare linear probing, CoOp, CLIP-Adapter, Tip-Adapter and the classic end-to-end fine-tuning method, ZS-init-CE on downstream task performance on 9 image classification tasks. Not only do we find end-to-end fine-tuning to consistently outperform adapters and prompt tuning, but when there is a big performance gap between the linear probe and end-to-end fine-tuning, adapters and prompt tuning do not bridge that gap well. This observation reaffirms the importance of end-to-end fine-tuning and thereby necessitates the investigation of better end-to-end fine-tuning methods for tasks where the pre-trained model's performance is sub-par. In Table in the appendix, we also show how concept forgetting in end-to-end fine-tuning methods used in our work can be further improved by combining them with Wise-FT.
Knowledge Distillation & Continual Learning: The LDIFS regularizer can be studied from the lens of knowledge distillation, an approach which has been applied to different ends like calibration He al. (2023), pre-training Lee et al. (2022). transfer learning Zhou et al. (2022c), robust fine-tuning Mao et al.
2022 and continual learning Li & Hoiem (2017); Jung et al. (2016): Rebuffi et al. (2017): Hou et al. (2018).
The main difference between these works and ours is that unlike classic distillation methods which mainly use the last layer features Jung et al.] (2016) or the logits Li & Hoiem (2017); Rebulli el al. (2017) directly for distillation, we concatenate features from shallower layers in the network when computing the LDIFS regularizer. Furthermore, while works like Zagoruyko & Komodakis (2016); Passban et al. (2021): ) Passalis &
Tefas (2018); Heo et al. (2019) develop indirect methods of distilling from intermediate feature representations
including additional attention layers Zagoruyko & Komodakis (2016); Passban et al. 20021 probability
12

---

Published in Transactions on Machine Learning Research (07/2024)
0.900
8:95
6.31
0.24
DFAR100
RES190
2:00
8 0:22
6:24
30.72
260
6.70
0.55
85 30 Training completion
25
0.37
20 16
20
103
20
10
103
20 60 103
40
Training completion
Training completion
Tranning completion
56 Training completion
Figure 10: ALP for models trained with LDIFS on EuroSAT using the full concatenated feature vector vs just the last layer (LL) feature vectors. Full feature vectors are crucial for LDIFS's performance.
distribution matching Passalis & Tefas 2018), and hidden ncuron activation boundarics Hoo et al. (2019). J
LDIFS uses L.2 distance between intemediate feature representations directly as the regularizer. This makes it simpler to implement without any additional layers to be trained on intermediate feature representations.
Finally, the proposed modifications to distillation in LDIFS turn out to be crucial for performance. To investigate the importance of distilling from earlier features, in Figure 10 we compare performance of LDIFS when distilling from just last layer features (we call this ablation LL) during fine-tuning on EuroSAT. These plots provide evidence that using just the last layer features is not nearly as performant as using including the earlier features in the feature vector. We provide further insights on this in Appendix C.S Finding which features contain pre-trained information for a downstream task is an interesting area of further research.
LDIFS vs LPIPS Zhang et al. (2018): One can find similarities between our proposed LDIFS regulariser and the LPIPS metrie Zhang et al. (0018 popularly used for measuring perceptual similarity between images. However, while LPIPS uses feature space distance on a pre-trained, frozen model to find perceptual similarity between pairs or sets of images, LDIFS instead feature space distance between a pair of pre-trained and fine-tuned model for the same image, to preserve the input-output behaviour of the pre- pre-trained model.
7 Conclusions & Remarks
We explore how end-to-end fine-tuning approaches often applied on foundation models can, in turn, significantly damage the model's ability to recognize concepts outside the fine-tuning task at hand, a phenomenon which we call concept forgetting. Such an effect is undesirable ble particularly for foundation models which have been specifically trained on millions of samples to encode information on a vast number of real-world concepts. However, we find that among fine-tuning methods, L2SP, which keeps the model near the pre-trained model in its parameter space suffers less from concept forgetting. From insights gained by analyzing L2SP, we also find that feature space distance provides a better definition for the vicinity of the pre-trained model as the pre-trained concepts are indeed encoded in the feature space. Our proposed new regularizer, LDIFS, encourages the features of fine-tuned model to be in the vicinity of the pre-trained ones. Through extensive evaluation on 10 diffcrent downstream classification tasks, as well as a continual fine-tuning sctup of 3 different sequences lasks each, we showed that LDIFS significantly alleviates concept forgetting without impeding downstream task performance.
Limitations & Future work Though we investigated concept forgetting and proposed promising fix, we believe that various other experiments and analyses along these lines would be valuable for the community. First, LDIFS has an associated hyperparameter ALDIFS which balances the objective of preserving pre- pre-trained model behaviour with the objective of fine-tuning on the task at hand. Following convention, we cross-validate this hyperparameter on a held-out validation set (see Appendix A from the task we fine-tune on. While this performs well, it interesting to see if there can be better ways of setting this hyperparameter and in which cases the cross-validated hyperparameter may not yield the best model either concept forgetting or downstream performance. Second, to compute LDIFS, we network. In this regard, an analysis of which features are useful for encoding which concepts is still open question worth investigating. Third, LDIFS tries to preserve the behaviour of the pre-trained model while fine-tuning. If there are biases and undesirable behaviours in the pre-trained model, these can percolate inlo the fine-tuned model. It is worth studying how such undesirable behaviours can be prevented from being distilled into the fino-tuned model while preserving the pre-trained knowledge. Fourth, understanding concept forgetting in other foundation model families such as Large Language Models, would be interesting. Works such as Jin et al. (2021): 3 Scialom et al. (2022) have already explored this direction to certain extent and it.
fixed set of features evenly spaced out in the
13

---

Published in Transactions on Machine Learning Research (07 (07/2024)
would be interesting to identify commonalities and differences in fine-tuning approaches between different foundation model families. Finally, fundamentally defining what "concept" is and the granularity at which we should study it for foundation models is important and requires research efforts to understand phenomena like "concept forgetting" beyond just performing based performance evaluation on a downstream dataset.
Acknowledgements This work is supported by the UKRT grant: Turing AT Fellowship EP RP/WW02981, / and EPSRC//MRI grant: EP/N019474/1. The authors would also like to thank the Royal Academy of Engincering and FiveAI.
References
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohgg, Antoine Bosselut, Emna Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXi::2108.0723 2021.
Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the Europeon conference on computer vision (ECCV), pp. 532-547, 2018.
Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 106(10):1867-1888 2017.
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Compuler Vision and Pallern Recognilion (CVPR), 2014.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference computer vision and pottern recognition, pp. 248-22282 255... Ieee, 2009.
Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. arXiv preprint ar xrx::2111000454. 2021.
Jiaxin Ge, Hongyin Luo, Siyuan Qian, Yulu Gan, Jie Fu, and Shanghang Zhan. Chain of thought prompt tuning in vision language models. arXiv preprint arXiv:334.00919, 2023.
Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. Finetune like you pretrain: Improved finetuning of zero-shot vision models. arXiv preprint arXic:2212.000 206388, 2022.
Guande He, Jianfei Chen, and Jun Zhu. Preserving pre-trained features helps calibrate fine-tuned language models. arXiv preprint arXiw:230.. 19249, 2023.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019.
Byeongho Heo, Minsik Lee, Sangdoo Yun, and Jin Young Choi. Knowledge transfer via distillation of activation boundaries formed by hidden neurons. In Proceedings of the AAAT Conference on Artificial Intelligence, volume 33, pp. 3779 3787, 2019.
Saihui Hou, Xinyu Pan, Chen Change Toy, Zilei Wang, and Dahua Lin. Lifelong learning via progressive distillation and retrospection. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 437 452, 2018
Gabriel Iaarco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongsook Namkoong, John Miller, Hannanch Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, jul 2021. URL http::///oii.ogg/10.5/1////
5143773
14

---

Published in Transactions on Machine Learning Research (07/2024)
Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannanch Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint orXin:2112.04088 2022a.
Gabriel Mharco, Mitchell Wortsman, Samir Yitzhak Gadre, Shuran Song, Hannaneh Hajishirzi, Simon Kornblith, Ali Farhadi, and Ludwig Schmidt. Patching open-vocabulary models by interpolating weights. arXiv preprint ar Xtx:2200 05592, 2022b.
Chao Jia, Yinfci Yang, Ye Xia, Yi-Ting Chen, Zarana Parckh, Hieu Pham, Quoc Lc, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning, pp. 4904-4916. PMLR, 2021.
Xisen Jin, Dejiao Zhang, Henghui Zhu, Wei Xiao, Shang-Wen Wen Li, Xiaokai Wei, Andrew Arnold, and Xiang Ren. Lifelong pretraining: Continually adapting language models to emerging corpora. arXiv preprint (xXi:22100088544 2021.
Hecchul Jung, Jeongwoo Ju, Minju Jung, and Junmo Kim. Less-forgetting learning in deep neural networks arXiv preprial ar Xrx 002 2016.
Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic lorgetting neural networks. In Proceedings of the AAAT conference on artificial intelligence, volume 32, 2018.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnicszka forgelting in neural nelworks. Proceedingg of the national academy of sciences, (15111):521 3526, 2017.
et al. Overcoming catastrophic
Will Knight. Openai's ceo says the age of giant ai models is already over. Wired, April, 17:2023, 2023.
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEFE international conference on computer vision workshops, pp. 554 561, 2013.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang, Fine-tuning can distort pretrained features and underperform out-of-distribution. arXiv preprint orXiv:200.. 10054, 2022.
Yann LeCun, Léon Botton, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Seanie Lee, Minki Kang, Juho Lee, Sung Ju Hwang, and Kenji Kawaguchi. Self-distillation for further pre-training of transformers. arXiv preprint arXiv:2210.028% 2022.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(1)):2 2945,,997444 2017.
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Tlya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint orXio:1711.0511 2017.
Xiaofeng Mao, Yuefeng Chen, Xiaojun Jia, Rong Zhang, Hui Xue, and Zhao Li. Context-aware robust fine-tuning. arXiv preprint arXiv:221.166115 2022.
Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pp. 109-165. Elsevier, 1989.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Ng. Reading digits in natural images with unsupervised feature learning. 2011.
15

---

Published in Transactions on Machine Learning Research (07/2024)
OpenAL. Chatgpt: An ai language model by openai. https://openai.com com 2020.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafranicc, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint KKW:22344001993, 2023.
Omiros Pantazis, Gabriel Brostow, Kate Jones, and Oisin Mac Aodha. Svl-adapter: Self-supervised adapter for vision-language pretrained models. arXiv preprint arXiv:2210.09794 2022.
Nikolaos Passalis and Anastasios Tefas. Learning deep representations with probabilistic knowledge transfer. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 268-284, 2018.
Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, and Qun Liu. Alp-kd: Attention-busel layer projection for knowledge distillation. In Proceedings of the AAAI Conference on artificial intelligence, volume 35, pp. 13657-13665, 2021.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapcau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2888 2011.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748 8763. PMLR, 2021.
SylvestroAAlvse Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Poltern Recognition, pp. 2001-2010, 2017.
Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. Fine-tuned language models are continual learners. arXiv preprint ar Xix:2205.12230, 2022.
Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. arXiv preprint arXiv:2209.07511 2022.
Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Wojeiech Galuba, Marcus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15638 15650, 2022.
Karan Singhal, Shekoofch Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis. Stephen Pfohl, al. Large language models encode clinical knowledge. arXiv preprint arXfo:2212.11113 2022.
James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonille Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Contimual decomposed attention- based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pottern Recognition, pp. 11909 11919, 2023.
Johannes Stallkamp, Mare Schlipsing, Jan Salmen, and Christian Tgel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks, 32:323-33 002,,33222220 2012.
Jingchen Sun, Jiayu Qin, Zihao Lin, and Changyou Chen. Prompt tuning based adapter for vision-language model adaption. arXiv preprint arXiv:2303. 15234, 2023.
Vishal Thengane, Salman Khan, Munawar Hayat, and Fahad Khan. Clip model is an efficient continual learner. arXiv preprint arXiv:2210.082: 03.01114, 2022.
Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. Xriio preprint 2rXi::2210.1000 2022a.
16

---

Published in Transactions on Machine Learning Research (07/2024)
Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Rucxi Sun, Han Zhang, Chen-Yu Yu Lee, Xiaoqi Ren, Guolong Su, Vincent. Perot, Jemnifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, pp. 631- 648. Springer, 2022b.
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Compuler Vision and Pollern Recognition, pp. 139 149, 2022c.
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roclofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pp. 20665-23998. PMLR, 17-23 Jul 2022a. URL https: //proceedings... longgsmmlz.. www.mm....essee. peress/v162/oortaann
html
Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannanch Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine- tuning of zero-shot models. In Proceedings of the JEFE/CVP Conference on Computer Vision and Pallern Recognition, pp. 7959 7971, 2022b.
LI Xuhong, Yves Grandvalet, and Franck Davoine. Explicit inductive bias for transfer learning with convolutional networks. In International Conference on Machine Learning, pp. 2825-2834. PMLR, 2018.
Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022.
Sergey Zagoruyko and Nikos Komodakis. Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv preprint urXiv:1612.03922 2016.
Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slea: Slow learner with classifier alignment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19148- 19158, 2023.
Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip- adapter: Training-free clip-adapter for better vision-language modeling. arXiv preprint arXiv:2111003990 2021.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xiangyu Yue, and Yang You. Preventing zero-shot transfer degradation in continual learning of vision-language models. ar preprint arXiv:2303.0663
2023.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the TEEE/CVE Conference on Computer Vision and Paltern Recoynilion, pp. 16816 16825, 2022a.
Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-languaae models. International Journal of Computer Vision, 130(9):2337 2248, 2022b.
Yang Zhou, Zhanhao He, Keyu Lu, Guanhong Wang, and Gaoang Wang. Preserve pre-trained knowledge: Transfer learning with solf-distillation for action recognition. arXiv preprint ar arXiv:2205. 200::2205000000, 2022c.
17

---

Published in Transactions on Machine Learning Research (07/2024)
In Appendix A we provide training details for each of our fine-tuning run. In Appendix B we provide details on how to compute the concatenated feature vector (x(x) from ViT-B/32 CLIP model for LDIFS. In Appendix C we provide additional results on i) observing how fine-tuning can lead to concept forgetting, ii) our investigations on why LP-init-L.2SP is a better baseline compared to others in avoiding concept forgetting and ii) performance of our proposed LDIFS regularizer.
A Training details
Training datasets and hyper-parameters: We fine-tune the CLIP ViT-B/32 model on each of the 10 image classification datasets: a) Stanford Cars, b) CIFAR-I0, c) CIFAR-100, d) DTD, e) EuroSAT, f) GTSRB, g) MNIST, h) RESISC45, 1) SVHN, j) ImageNet. For cach datasct, we have a separate fine-tune run for each of the baselines discussed in 2 For each run, we train using the AdamW Loshchilov & Hutter (2077) optimizer, with an initial learning rate of 1e 5, weight decay of 0.1 and a cosine learning rate scheduler with warmup length of 500. For all the runs, use batch-size of 128. Following the code of Iharco et al. (2022a), we use the following number of epochs to fine-tune cach dataset: a) Stanford Cars: 35 epochs, b) 10/AR-101100 100: 10 epochs, c) DTD: 76 epochs, d) EuroSAT: 12 epochs, e) GTSRB: 11 epochs, f) MNIST: 10 epochs, g) RESISC45: 15 epochs, h) SVHN: 10 epochs and j) ImageNet: 10 epochs. We keep a minimum of 10 epochs for fine-tuning.
Compute and number of experiments: Each of our fine-tuning runs is done on a single NVIDIA A100 GPU. As there are a total of 10 classification datasets and 8 finc-tuning basclines (including our proposed LDIFS), we perform total of 80 fine-tune training runs from pre-trained CLIP ViT-B/32. Furthermore, in order to produce the plots capturing ALP, as well as & distance in parameter and feature space, we store intermediate checkpoints over the course of fine-tuning. Particularly, for each run, we evaluate checkpoints after every 20% of training completion. Finally, in order to obtain the full set of results in Table [ and Table g for the 72 models fine-tuned on datasets other than ImageNet, we evaluate each of them on 9 datasets (i.e., including the test set of the fine-tuned task), thereby having a total of 648 evaluations. For the 8 models fine-tuned on ImageNet, we evaluate them on a total of 8 datasets (leaving out CIFAR-10/ 10/000,, thereby totalling 64 evaluations. Thus, we have total of 648 + 64 = 712 evaluation runs. Similar to training, each evaluation is also performed on single NVIDIA A100 GPU.
which the LDIFS regularizer introduces is ALDIES A higher value of ÀLDIFS encourages the model to preserve features of the original foundation model and vice versa. For each classification task, we performed a grid search over ALDIFS E {0.01, 0.05, 0.5, 1, 10, 100) and cross-validated this hyper-parameter on a held-out validation set, choosing the value which produces the best performance the validation set. We found ALDIFS 10 to produce the best performance over datasets in general, so all the results we present in this paper are with ALDIFS set to 10.
Choosing ALDIFS: One
B Computing (x)
In this section, we discuss how we compute the concatenated feature vector 4x(x) given input and model parameters 0, specifically for a VIT-B/32 model. This is used for computing LDIFS, our proposed regularizer for fine-tuning.
Let the feature output from layer in the network for input x be (x) E R Then the normalized feature (x)) In order to form the concatenated feature vector, one can take technically take features from every intermediate layer of the network. However, storing all the features is memory intensive. Thus, we follow similar approach to the LPIPS Zhang et al. (2018) implementation for VGG and AlexNet, and choose 5 intermediate points in the VIT-B/32 architecture to collect features from. For a single input image x E IR3x2242221 each of the intermediate feature representations has a dimension of 768 with 50 tokens and 768 dimensional representation for each token. When normalizing the feature vector, we flatten this vector out to single 38400 dimensional vector. Thus the full concatenated feature vector (x(x) has dimensionality 5 x 38400 192000. However, note that there can be other ablations to this design and we leave that for future exploration.
vector for layer can be represented as [[
18

---

Published in Transactions on Machine Learning Research (07/2024)
C
Additional results
In this section, present additional empirical results to supplement our observations and conclusions in the main paper.
C.1 Concept forgetting with Azs and Azs
we proposed a simple method to quantify concept lorgetting using the difference in LP accuracy ÂLP between the pre-trained and fine-tuned models on task. We also discussed how using ZS accuracy Azs in the same way may not be suitable, as ZS linear classifiers may not be indicative of model recognizing concepts post fine-tuning. Nolethe for the sake of completeness of our analysis, in the additional results we present below, we also include Azs and compute A2s as:
In Section 3.1
Ass (D. for fo Azs(fo, Azs(fo. D).
Similar to ALP, a negative Ass would indicate concept forgetting, a wero would indicate knowledge accu- mulation and positive value would indicate knowledge gain or positive forward transfer on the task under inspection.
C.2 Crippling effect of fine-tuning
In 1 we observed how most existing funi-tuning methods, while gaining state-of-the-art performance on fine-tuned tasks, can lead to concept forgetting models. In this section, we provide additional empirical results further strengthen those obsorvations.
In Figure 11 and Figure 17 we present the Azs and ALP accuracy respectively, for models fine-tuned 9 classification tasks, their respective test sets. This shows expected behaviour models broadly achieve very high test set accuracy, which increases over the course of fine-tuning, on their respective fine-tuned tasks. Interestingly, LP-init. baselines seem to have relatively lower performance on Azs accuracy compared to their ZS-init counter-parts However, this reduced performance is only limited to Azs as this does not translate to a reduced performance ALP. Moreover, L2SP baselines (both ZS and LP-init) obtain relatively lower fine-tuned accuracy both in case of Azs and ALP.
In Figure 12 and Figure 18 present the Azs and ALP for models finc-tuned EuroSAT (first row), GTSRB (second row) and SVHN (third row) captured on 8 classification datasets different from their respective fine-tuned set. Broadly, the ZS and LP performance for all models drops on other datasets as they are fine-tuned, thereby capturing concept forgetting in the fine-tuned models. Among the baselines, we consistently observe LP-init-L2SP to perform better than others in avoiding concept forgetting. This is evident through the distinctly higher Azs and AL. accuracies over fine-tuning for the I.P-init-L2SP baselines.
C.3 Analysing L2SP
In this section, provide additional results to supplement our observations related to investigating the L2SP baseline in 4 of the main paper. In Figure 21 we present the & distance in parameter space ||0- 0 between the pre-trained and current models captured over fine-tuning. For all the datasets, we observe that L.SP baselines while initially diverging slightly in the parameter space, converge back to a model having low l parameter space distance from the original foundation model. Other basclines on the other hand, completely diverge away from the original model in the parameter space.
To further investigate the change in input-output behaviour of the model over fine-tuning, we measure the distance in feature space (see Equation (3) over fine-tuning. In Figure 22 we present the & distance in feature space captured for models fine-tuned on EuroSAT. Again, consistent with our previous observation, we find that unlike other baselines, L2SP first diverges away from the original foundation model and then converges back to the original input-output behaviour, as indicative through a decreasing I.2 feature space distance in the later stages of fine-tuning. This observation is consistent on the EuroSAT train set, the EuroSAT test set well as other datasets, thereby providing our motivation for the LDIFS regularizer.
19

---

Published in Transactions on Machine Learning Research (07 (07/2024)
(a)
(b)
(c)
(d)
(e)
(g)
(h)
(1)
Figure 11: Test set ZS Accuracy Azs for different fine-tuning methods on image classification
tasks: (a) Stanford Cars, (b) CIFAR-10, (c) CIFAR-100, (d) DTD, (e) EuroSAT, (f) GTSRB, (g) MNIST (h)
RESISC45 and (i) SVHN. Azs generally rises over the course of fine-tuning. However, for the ZS-init-L.2P and LP-init-L2SP baselines, the gain in Azs is relatively lower. Furthermore, for LP-init baselines, the performance is consistently lower compared to other baselines
(b)
(d)
(e)
(h)
Figure 12: ZS Accuracy Azs for models finc-tuned on EuroSAT (first row), GTSRB (sccond row), and SVHN (third row) and evaluated on 8 different datasets different from their fine-tuning dataset. Most fine-tuning methods show a drop in Azs performance over the course of fine-tuning indicating a reduction in the model's transferability.
C.4 Performance of LDIFS
In this section, we provide additional results to supplement. observations related to the performance of the proposed LDIFS regularizer.
Firstly, in Figure 15 and Figure 19 we present the test set accuracy of all fine-tuning methods including LDIFS on all 9 classification tasks. We see that LDIFS performs competitively with all other baselines and improves on L2SP consistently. In Figure 16 and Figure 20 we present the Azs and ALP accuracy respectively, for models fine-tuned on EuroSAT (first row), GTSRB (second row) and SVHN (third row) over the course of fine-tuning. Furthermore, we also provide the Azs, ALP and the ALP values for fully fine-tuned models 9 classification tasks and 8 baselines in Table
As is clear from our results, LP-init-LDIFS consistently minimizes concept lorgetting without compromising on performance on the fine-tuned task. This is evident from its high Azs, ALP (see Figure 16 and Figure 20) and high Azs and ALP (see Table g when evaluated on other tasks. Note from Table that even when LP-init-LDIFS is not achieving the best performance on a certain dataset pair, it is often close second with very little difference compared the best performing baseline. In addition this advantage, its Aur accuracy on the fine-tuned task itself very competitive with the top scores obtained by other fine-tuning basclines and provides a consistent improvement on L2SP.
C.5 Comparison with Adapters and Prompt Tuning
In Table 8. we present results of fine-tuning a CLIP ViT-B/32 model using linear probing, CoOp, CLIP- Adapter, Tip-Adapter and full end-to-end fine-tuning using 7S-init-CE. loss 9 image classification tasks.
20

---

Published in Transactions on Machine Learning Research (07/2024)
We consistently observe end-to-end fine-tuning to produce the best test set accuracy across all 9 tasks. Furthermore, we also note that in case of tasks where the gap between linear probe performance and end-to-end fine-tuning performance is significant (e.g. SVHN), adapter and prompt tuning approaches don't cover this performance gap. Hence, if the pre-trained encoder is not performant on a downstream task, the only way lo achieve stale-of-the-arl results on the task requires end-lo-end fine-tuning the model on the task itself. This observation thus necessitates study into better end-to-end fine-tuning methods for foundation models.
C.6 Comparison with Wise-FT
In this section, we perform an ablation to see the effect of Wise-FTT Wortsman et al. (2022b) on preventing concept forgetting. Wise-FT weight interpolation method which forms linear combination between the pre-trained 0g and fine-tuned 0 model parameters:
wwee atly + a
(7)
Since this can work with any fine-tuned model, we can apply it on all the end-to-end ine-tuning baselines in this work. Specifically, we tune the hyperparameter a on a held-out validation set of the downstream task hand to maximum validation accuracy. We evaluate concept forgetting on downstream tasks: CIFAR-10, EuroSAT and SVHN using the ALP metric and report the results in Table 7 For each dataset, evaluate the mean ALP on 5 other downstream tasks: Cars, DTD, GTSRB, RESISC45 and MNIST. Our obscrvations show a consistent reduction in concept forgetting across all fine-tuning baselines. However, the order of performance between the fine-tuning baselines does not change and LDIFS still maintains superior performance over other baselines both pre and post application of Wise-FT. Thus, the results show that Wisc-FTT can be combined with any E2E fine-tuning method to further minimize concopt forgetting.
C.7 Additional results on continual fine-tuning
A Ablation with joint training. Joint training Li & Hoiem (2017) is a continual learning baseline often used to upper bound the performance of continual learners. The idea is to use samples from different tasks in a manner that minimizes forgetting. In this experiment, we take the simple setting of joint training, where each step of the sequence when new dataset comes in, we retrain the model on the combination of all the training data from earlier datasets in the sequence along with the new incoming data. The training is done at each point using the ZSS-nnit-CB CE loss function, which is the vanilla go-to fine-tuning method.
In Table 5 we compare results of the joint training bascline with LP-imiitLDIIFS. Note that ALP is computed using Equation (5) where we compule forgetting on a lask in the sequence as the difference between the model fine-tuned on the entire sequence and the model in the sequence which performed the best (had the most knowledge) on the task. We can make the following obscrvations from the results:
1. Earlier tasks in the sequence: On tasks seen earlier in the sequence, (like SVHN), LP-init-LDIrs is competitive but slightly underperforms compared to joint training which sees the full training set comprising all previous tasks at every point in the sequence.
2. Current/ Last task in the sequence: On the final task in the sequence, ie., RESISC45, LDIFS performs competitively with the joint training baseline.
3. Tasks not in the sequence: On tasks not seen in the fine-tuning sequence, LDIFS significantly outperforms joint training exhibiting its ability to minimize concept forgetting.
Comparison with continual lcarning basclines that use pre-trained encoders. While the motivation of the work lies in fine-tuning foundation models on downstream lasks, and in their eventual continual fine-tuning, the proposed LDIFS method is general enough to be applied in standard continual learning setups as well. Furthermore, in the continual lcarning literature, there are works like I.P Wang et al. (2022c). DualPrompt Wang al. (2022b), CODA-Promps Smith et al. (2023), Continual- CLIP Thengane et al. (022) which utilize pre-training in order to improve performance well-known continual learning
21

---

Published in Transactions on Machine Learning Research (07/2024)
Contioual Leaening LP-inttLLDISS (Ours)
SVHN C10 R45
RESISONS
CIPAR1O
5.88
05.19
3.84
95.0
Others
-1.4
80.13
67.05
SVHN C100 R45
RESISC45
CIPAR100
0.05
3.86
95.07
3.83
-0.5
66.51
80.11
Others
-1.67
80.62
-0.25
89.12
SVIN
-0.11
13
-0.17
85.9
SVHN Cura R45
RESISC45
Cara
0.82
3.83
95.24
$4.12
0.47
3.83
00.27
54.23
Othes
4.35
83.42
0.28
89.89
Table 5: ALP and ALp% comparing LDIFS with joint training as CL baseline on the 3-task setup.
Average uccuracy Contimuil-CLIP
12P
DualPrompt
CODA-Prompt
SLCA
L.P-init-I.DIPS (ours)
74.6 + 1.21
17.24 1.27
78.13 + 1.18
76.23 + 118
81.22
1.29
83.02 1.16
Table 6: Performance of continual learning baselines on Split ImageNet-R.
2 2.003
0.604
9175
0150
6.325
TPP
1.000
0.000
0030
0.006
S0.00
0030
2-DCCE
@ 0.00
0023
9. 0023
0.0000
000000
% Training completion
%6 Training completion
56 Paining completin
55 Trannnn completion
TTh Thining compets
Figure 13: l2 distance in between features obtained from a pre-trained model and model fine-tuned EuroSAT. The last-layer ablation for LDIFS does not minimize feature space distance compared to using earlier features well.
benchmarks. While we don't target our method specifically lowards such benchmarks, in this section, we provide comparison Split ImageNet-R Wang et al. (2022b), well-known class-incrementa, learning benchmark. Specifically, we compare with the other methods mentioned above which use pre-training as part of their method to improve continual learning. We present results in Table D clear from the results that L.P-initL.DIFSS outperforms competitive baselines indicating its usability in classic class-incremental learning benchmarks as well.
C.8 Last Layer Ablation
In this section, we present some results to provide additional insights on why LDIFS using just the last layer does not perform as well as using the full feature vector. To see this, we plot the feature space distance d(0,0p, 0g , D) for all baselines including LDIFS well L.-init-LDDIFS with just the last. layer for models fine-tuned EuroSAT. The plots shown in Figure 13 These plots show that the full feature distance is not minimized if we just use last layer feature difference in the LDIFS regularizer, thereby indicating that learned concepts can indeed be encoded in shallower layers of the network which makes distilling from these layers crucial.
C.9 Visualizing Table 1
In this section, we present the results of Table 1 using alternate visualization. For 9 downstream datascts, we plot the LP accuracy on the dataset itself on the x-axis and the average LP accuracy on all other datasets on the y-axis. The best baselines should lie on the top right corner of this plot as that indicates that as the model fine-tuned on the dataset, still retains performance on other datasets. Results are Figure 14 Clearly, we consistently see the LDIFS baselines lic the top right, again indicative of their performance in not just minimizing concept forgetting but also obtaining excellent downstream task performance.
22

---

Published in Transactions on Machine Learning Research (07 2224)
20.00.1
35.0 935 260 15. 913
24
74
0.6
1204
Scorssy OUESAT
Scouecy 57513
Acanso FRIST
Act Accance R250C45
70 25. 50
Figure 14: Visualizing test LP accuracy of fine-tuning baselines on different datasets. For each dataset, plot its test set LP accuracy the x-axis and the average test set LP accuracy of the remaining datasets on the y-axis.
Distasst
PLYP
FLYRCE
LP-int-- (UP-PT))
65chii gap
25-imi-LLDPP (Oun) LP-Stt-LDDPS (Chrrs)
Steldonel
Aup
Orelone:
Aie
Dir
Suplomee
Aee
Arat
Au
Stelimeet
Sie
CIEARIn
5.41
1.31
11
61
-1 1.40
1.88
1:44
1.0
1.00
1:8
1:0
1.92
2.81
SUEN
1088
-030
1094
-358
1000
-315
-033
9,07
015
633
Table 7: ALP without and with Wise-IT Wortsman et al. (2022b) for all fine-tuning baselines on CIFAR-10, EuroSAT and SVHN.
10
Figure 15: Comparing test set ZS Accuracy Azs for different fine-tuning methods with LDIFS image classification tasks. While LP-init baselines generally underperform Azs, find ZS-init-LDIFS to be competitive with the best baselines.
Dastsse
LPP
CoOp
CILIP%Adppter
Tip-Adapter
F/R Finestaming
$0.50
80.85
$1.24
CIFAR1O
94.92
95.02
94.87
95.13
73
CIFAI210
75.03
80.12
40.00
80 57
NA. it
DTD
72.08
72.00
71.87
72.02
18
MNIST
SVEN
63.47
67.25
60.78
60.23
67.4
Table 8: Test set accuracy obtained from linear probing (LP), CoOp, CLIPe Tip-Adapter and end-to-end (E2F) fine-tuning using 7S-init-CE, loss.
23

---

Published in Transactions on Machine Learning Research (07 (07/202) 2024)
Turegerendss
(a)
(b)
(c)
(d)
(f)
(g)
(h)
Figure 16: Comparing ZS Accuracy Azs of different finc-tuning methods with LDIFS for models fine-tuned on EuroSAT (first row), GTSRB (second row) and SVHN (third row) and evaluated on 8 datasets different from their fine-tuning dataset. AP-init-LDIFSS almost consistently beats all other baselines including LP-init-L2SP in preserving the model's transfcrability.
(a)
(b)
(c)
(d)
(e)
(h)
(i)
Figure 17: Test set LP Accuracy ALP for different finc-tuning methods on image classification
lasks: (a) Stanford Cars, (b) CIFAR-10, (c) CIFAR-100, (d) DTD, (e) EuroSAT, (f) GTSRB, (g) MNIST (h)
RESISC45 and (i) SVHN. ALP generally rises over the course of fine-tuning. However, for the ZS-init-L2SP and LP--nit-L22PP basclines, the gain in ALP is relatively lower. Furthermore, for LP-init basclines, the performance is consistently lower compared to other baselines
19
Taiing
(b)
(c)
(d)
(e)
(h)
Figure 18: LP Accuracy ALP for models fine-tuned EuroSAT (first row), GTSRB (second row), and SVHN (third row) and evaluated 8 different datasets different from their fine-tuning dataset. Most fine-tuning methods show drop in ALp performance over the course of fino-tuning indicating reduction in the model's transferability.
24

---

Published in Transactions on Machine Learning Research (07 (07/2023) 2024)
6M
Figure 19: Comparing test set LP Accuracy ALp for different fine-tuning methods with LDIFS on 9 image classification tasks.
18
6.01
Thaiigg corsetor
5 siaaggoooiin
3710
2010
2ca
Qun
6.59
(a)
(b)
(c)
(d)
(e)
(g)
(h)
Figure 20: Comparing LP Accuracy ALP of different fine-tuning methods with LDIFS for models fine-tuned EuroSAT (first row), GTSRB (second row) and SVHN (third row) and evaluated on 8 datasets different from their fine-tuning dataset. LP-init-LDDI'S almost consistently beats all other baselines including LP-init-L2SP in preserving the model's transfcrability.
0.50
Tarey
Figure 21: l distance in weight space 00010112 between pro-trained image encoder foa and finc-tuned image encoder fo over the course of fine-tuning. Apart from ZS-init-L2SS and LP-init-L2S5 all fine-tuning baselines diverge in weight space over the course of finc-tuning.
Figure 22: l distance in feature space d(A, 00,,0,,20 D), between fino-tuned and pro-trained image encoders, fo and Sog computed over different finetuning methods for models fine-Luned on EuroSAT.
25

---

Published in Transactions on Machine Learning Research (07/ 2224)
Awitt
baitt
Beall
Aucti
built
Buell
buelt:
Bull)
Aufl
Axll. | Aurt)
Aulf
Arll
[Surh)
10.35
1348
-42
+1.1
-3.0
-330
11.0
+2208
41
+3.7
01
+123
-8.5
8.13
15.5
100
68.33
+1.1
06.0
+2.8
+0.06
15.15
-1.03
+3.01
$1:47
-57
13.0
116
+3.11
03.01
1.5g
18.23
1023
1113
F1.
1.50
-4.13
18.10
66/6
1.2
6.06
$6.17
1.42
61.4%
300
4.1%
04.00
9.44
145
01.44
-341
0.80
01.17
$1.00
4.1
02-
16.34
6516
7.6
36.15
2.55
01.01
28.58
8.75
11.03
0.33
01
4.13
0.15
08.05
11:35
8.55
0/
2.30
$3.50
+1.0
8635
11
1667
9933
31
-3.4
0037
11/13
-1.0
913
-138
-1.00
0.27
0.85
1.80
3.13
12.3
1:45
1231
14
17.35
9.30
08.13
155
5.55
17.11
4.16
07.
0.56
3.5g
17.50
219
8:00
27.50
-.33
1225
13.5
3.36
17.11
23
3.55
92.55
13.3
1.35
90.0
0.76
6.33
1.33
315
13.38
3.75
8.13
1.00
15.33
5425
-1.0
0.95
$1.35
-3.3
-13
0-11
011
-4.00
1.8
6.01
-211
230
6.31
5.36
$0.05
1779
-3413
1125
-2.5
140
1.31
01:1
-0.3
1.41
-1510
1.75
325
05
-041
5.33
-0.2
1.25
1369
-81
8.31
$2.05
-3.0
-3.81
-3.44
615
11
6.70
-11.0
-12
18.99
-04.4
3.65
-1505
1.4
1/50
-153
1.2
-15
133
2.25
1.50
55.5
51
1503
-24
0.6
41:1
14
3.15
0030
4.65
156
1.50
1.29
1.5
313
6.76
23
4.07
$.36
$5.55
$5-11
1465
$1.00
$55
33.37
$170
3..0
3.35
$9.97
-3.15
3.31
25.11
3.81
0-2
3.80
$8.24
19.20
01.
0.30
1133
1921
2.42
-1.83
18.96
-3.75
-123
-1.5
-0A4
1.31
1.95
$1.51
19.10
1.35
-31.0
43.45
-1.34
13.41
45.35
3.24
15.00
01.05
-3.11
14.13
41
-11.0
1.6
3115
3.99
1.30
19.6
11.47
13.86
06.38
$1.
0:1
$1.33
18:43
01.51
01.4
47.4
05.27
-12.3
6164
-3.3
-3445
$1.55
02:55
-1811
$4.02
-0.04
-3.0
1..00
-2:4
89 50
10.00
-1.0
60.00
30000
01
+1
1.5
$034
-0415
02
4300
18.
3.5
3.30
$4.24
-1.11
15
1331
145
4.21
13
1.6
-5.29
8641
3.2
8.3
107
716
133
133
1330
1.55
1376
13.5
1379
+382
1034
-2255
-33
3334
-2-11
-24.3
+1
07A
-25.6
44.3
-1248
11.3
-1.8
15.1
12.0
-11.0
9224
-11
+151
11.14
4350
1137
01
-431
010
4131
-128
-3.4
$1.77
6.0
3.6
1141
3130
35.09
505
31.
31.0
915
3132
83
13.8
3141
102
1130
1836
Table 9: Azs, Air and Air for models fully fine-tuned on 9 different classification datasets. T.P. init-LDIFS outperforms other fine-tuning baselines and even LP-init-L2SP in minimizing concept forgetting, while also outperforming LP-init-L255 L25i on the fine-tuned task performance.
26


Reviews:
[{"review_id": "Lyd0vGaWaI", "reviewer": "Reviewer_zv3Y", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1) The paper is well written and easy to follow.\n2) The experiments with CLIP ViT-B/32 are thorough with a variety of downstream tasks and results.\n3) Paper presents an intuitive and easy to follow LDIFS  concept with thorough discussion on the method technique.\n\n\nCons:\n1) The motivation of usage of LDIFS models is not strong, paper goes deep in CLIP model but wants to generalise findings to other foundational models without evidence.\n2) Authors need to provide more evidence to support full fine-tuning can cripple your foundation models. There are methods such as joint training and others as provided in the comments below by fellow reviewers. How do those methods compare with LDIFS? Can we achieve similar results?\n3) The paper does not go in depth of weaknesses of  LDFIS.Where and when could it hinder performance? Is that dependent on model size or data or other parameters?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper investigates \"concept forgetting\" in foundation models, where fine-tuning on specific tasks degrades the model's ability to recognize concepts outside those tasks. It introduces a fine-tuning method, LDIFS (ℓ2 distance in feature space), designed to preserve pre-trained knowledge while learning new concepts. LDIFS minimizes the feature space distance between the original and fine-tuned models, significantly reducing concept forgetting. Extensive experiments across 10 tasks demonstrate LDIFS's effectiveness in mitigating concept forgetting compared to traditional methods. Additionally, in a continual fine-tuning setup, LDIFS shows promise in minimizing forgetting and facilitating positive forward transfer, where learning new tasks enhances performance on previously learned ones. The study highlights the importance of maintaining foundational knowledge in fine-tuned models and presents LDIFS as a viable solution to concept forgetting, marking a significant contribution to the field.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) Either focus more on CLIP and motivate the paper or conduct experiments of variety of foundation models\n\n2) The long term learning impact of continual learning. If extensive sequences are used then would there be cumulative biases or errors introduced? (Nice to have)\n\n3) A discussion on weaknesses of LDIFS\n\n4) Comparison with commonly used data training techniques such as joint training with pre/finetuning data.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "GMGqjTTuZg", "reviewer": "Reviewer_UAdC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n - The paper addresses an important problem of “concept forgetting” of foundation models.\n - The paper is well written and easy to understand\n\nWeaknesses:\n - **Novelty of Concept Forgetting**: The concept forgetting phenomenon seems very similar to catastrophic forgetting. Although the authors have tried to differentiate the two terms, they both seem the same to me. The authors define concept forgetting as  the drop in performance on a dataset different from the pre-trained dataset in foundation models. However, this also falls under the purview of catastrophic forgetting and has been explored by earlier works ([1], [2], [3], etc). Hence, I am not convinced that concept forgetting is a new phenomenon. Additionally, the authors need to compare their results with these approaches as well.\n - **Section 3.1: Subsection - Catastrophic forgetting vs concept forgetting**: First of all, it is very confusing when it is told that in the pre-foundation model era, the pre-training tasks were fully supervised. Yes its true. But this is also true for the so called foundation model CLIP that the authors have used as the to-go model. Going by the traditional definition of full supervision i.e., using input example and corresponding label/annotation, CLIP pretraining is also fully supervised as it uses paired image-caption data. The training uses the information of which caption is for which image and is not oblivious to it. So, taking away the factor of the 'use of supervision while pretraining', the subsection boils down to justifying the difference between catastrophic and concept forgetting as whether or not the pretraining dataset is small or big. Probably, this is a major source of confusion to some of the lines written later in the subsection - e.g., \"it was possible to roughly quantify the degree of damage ... on a new downstream task\". Why is it 'roughly' quantifiable? Then why is not 'trivial' for the foundation models. I can understand that the pre-training dataset is not often accessible. In absence of the pre-training dataset, the obvious choice would be to set aside a proxy dataset on which the effect of finetuning can be observed. And precisely this is done by the authors (the use of a target dataset). But why does having a target dataset in absence of the pretraining dataset need to rename catastrophic forgetting to concept forgetting? It is still forgetting and frankly speaking, if one does not have the original pretraining dataset, there is no way to tell precisely if the 'concepts' are forgotten or not. This is because, what all concepts the foundation model learnt in the first place can't be known or tested. So, in brief, why the term 'concept forgetting' is being differentiated with 'catastrophic forgetting' is not clear. c. Incremental Novelty: The paper proposes an approach similar to knowledge distillation which has been explored before. LwF [10 in main paper] and ICARL [11 in main paper] are two classical continual learning papers which have used such approaches. While the authors apply the feature regularisation over all layers as compared to final layers by knowledge distillation, the novelty seems incremental.\n - **More results: Section 5**: The authors showed comparisons on similar Continual Learning (CL) Datasets but not the same ones. Why not use the same datasets as is generally followed in CL literature - say 5-datasets or ImageNet-R or CUB etc. Additionally,  there exists prior works applying Continual Learning on CLIP models [4, 5, 6, 7 ]. The authors need to compare with them.\n - **Doubts on Claim**: The claim that full fine-tuning can cripple your foundation model is somewhat misleading as works like SLCA[8] and FSA[9] has proven otherwise. The authors need to show results with these approaches as well.\n\nReferences:\n\n1. Smith, James Seale, et al. \"Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.\n2. Wang, Zifeng, et al. \"Learning to prompt for continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022.\n3. Wang, Zifeng, et al. \"Dualprompt: Complementary prompting for rehearsal-free continual learning.\" European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.\n4. D'Alessandro, Marco, et al. \"Multimodal Parameter-Efficient Few-Shot Class Incremental Learning.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023\n5. Thengane, Vishal, et al. \"Clip model is an efficient continual learner.\" arXiv preprint arXiv:2210.03114 (2022)\n6. Chen, Haoran, et al. \"Promptfusion: Decoupling stability and plasticity for continual learning.\" arXiv preprint arXiv:2303.07223 (2023)\n7. Zhou, Da-Wei, et al. \"Learning without forgetting for vision-language models.\" arXiv preprint arXiv:2305.19270 (2023)\n8. Zhang, Gengwei, et al. \"Slca: Slow learner with classifier alignment for continual learning on a pre-trained model.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023\n9. Panos, Aristeidis, et al. \"First session adaptation: A strong replay-free baseline for class-incremental learning.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. \n10. Li, Zhizhong, and Derek Hoiem. \"Learning without forgetting.\" IEEE transactions on pattern analysis and machine intelligence 40.12 (2017): 2935-2947 \n11. Rebuffi, Sylvestre-Alvise, et al. \"icarl: Incremental classifier and representation learning.\" Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2017 \n12. Radford, Alec, et al. \"Learning transferable visual models from natural language supervision.\" International conference on machine learning. PMLR, 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the phenomenon of \"concept forgetting\" during fine-tuning of foundation models especially CLIP, where the fine-tuned model loses its ability to recognise concepts outside of the downstream task it was fine-tuned on. It proposed a new regularizer called LDIFS (l2 distance in feature space) that minimises distance between the original and fine-tuned models in feature space that seems to tackle the forgetting problem effectively.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please refer to the weaknesses above. Especially the last three points are related to this.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "pse5u6ihhS", "reviewer": "Reviewer_FFde", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n\n1. The LDIFS idea is intuitive and effective in addressing the concept drifting problem.\n2. The author conducted experiments on a diverse collection of datasets.\n\n\nCons:\n\n1. LDIFS is a type of continual learning algorithm but author has not compared with a well-estabilished baseline called continual learning with rehearsal. The idea of continual learning with rehearsal is to finetune the model on the combination of down-stream samples and a subset of general-domain samples (called rehearsal memory). It has been shown to be beneficial for finetuning instruction-tuned language models in [Fine-tuned Language Models are Continual Learners](https://aclanthology.org/2022.emnlp-main.410.pdf).In fact, how to continually adapting a foundation model has been studied in other papers such as [Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora](https://aclanthology.org/2022.naacl-main.351.pdf). The LDIFS algorithm is relevant to previous works that apply knowledge distillation in the feature space for continual adaptation. \n\n2. The author only compared different algorithm with impact of continual finetuning with ViT-B/32 version of CLIP. It is unclear if the same conclusion will transfer to models with different sizes. In fact, the transfer learning capability of large-scale models is relevant to the size (see [Scaling Laws for Transfer\n](https://arxiv.org/abs/2102.01293) ). It is unclear if the concept forgetting problem will be alleviated or worsen for larger-scale CLIP models.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper studied the \"concept forgetting\" phenomena in finetuned foundation models. After trained on a specific task, the finetuned models often lost their ability to recoganize general-domain concepts. To tackle the concept forgetting problem, the author propose to regularize the finetuning process by minimizing the L2 distance between features from the original model and the finetuned model (called LDIFS in the paper).\n\nIn the experiments, the author showed that the concept forgetting phenomena, which is measured by difference of linear probing accuracy, has a strong correlation with feature-space distance. The proposed LDIFS regularizer is effective in addressing the concept forgetting problem.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Comparison with other continual learning algorithms\nComparison with CLIP in different sizes to see how scale up the model size will impact concept forgetting.", "broader_impact_concerns": "No", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "WZKOqstOHz", "reviewer": "Reviewer_2J6R", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Pros:\n1. The proposed method to address the issue is straightforward and intuitive.\n2. The paper is well presented and overall easy to follow.\n3. The empirical analysis is interesting. \n\nCons:\n1. The major concern of the reviewer lies in whether the identified problem and setting is really important in practice or not. In practice, when one plan to fine-tune a foundation model on a downstream dataset, the domain of the application and the concepts that are important are usually already defined. For example, if the downstream application requires ability on understanding both satellite images and remote sensing, one can easily improve the model's ability on these domains by using joint training data from these two domains. It is also questionable whether the proposed method can address possible problems in this practical cases where there could be certain conflicts in the desired sets of concepts in the application.\n\n2. The current empirical analysis only covers one pretrained model. It is important to show that the method generalizes to different model architectures, and especially different scales of models in terms of both pretraining data and model size. Moreover, despite the popularity of CLIP, there are also vision only pretrained models such as MAE. It is also important to udnerstand whether the identified problem and the proposed method is generalizable enough to handle different pretraining methods.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses a critical issue in the fine-tuning of pre-trained foundation models: \"concept forgetting,\" where the fine-tuned model's ability to recognize diverse concepts diminishes. The authors observe that while fine-tuning on specific datasets improves performance on downstream tasks, it often reduces the model's pre-trained knowledge on other domain of concepts, a phenomenon they term \"concept forgetting.\" To counteract this, they propose a new fine-tuning regularizer named LDIFS, which is designed to preserve pre-trained knowledge while acquiring new concepts. Extensive experiments across 10 fine-tuning tasks demonstrate LDIFS's effectiveness in reducing concept forgetting and its superior performance in continual fine-tuning compared to standard fine-tuning and continual learning baselines.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please check cons for details.", "broader_impact_concerns": "It will be a plus if the authors can also apply the method in fairness settings.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "yWJRq4XgDW", "reviewer": "Reviewer_RQ9w", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Overall, the paper is well-written and easy to follow. Especially, Figure 1 and introducing the proposed method, LDIFS from the analyses of L2SP justify using linear probe (LP) for measuring concept forgetting and the validity of LDIFS, respectively.\n- Extensive experiments help readers understand the newly defined problem, \"concept forgetting\", and validate the effectiveness of LDIFS.\n- Enormous computational resources are utilized for experiments.\n\nWeaknesses\n- Editorial comments: \n  - TMLR can merge the Appendix just after the main paper, which would enhance readability.\n  - The analyses of LDIFS on parameter space and feature space distance, Figure 11 and Figure 12 in the Appendix, seem to be critical for deeply understanding the regularizer. Please move the analyses to the main paper, and also add discussion about why applying the regularizer on the target dataset also minimizes feature space distance, hence concept forgetting on other datasets (utilizing features from intermediate layers might be one reason).\n- With regard to the above comment, further discussion is needed for validating the design choice made for LDIFS: The authors should provide not only the delta LP values for LL (using the last layer only) but also the actual feature space distance on non-target datasets to clarify that the improvement in concept forgetting comes from reduced feature space distance, as the authors intended.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "I summarize the contributions of this paper as follows:\n1. The paper addresses a particularly practical issue in an era where foundation models pretrained on massive-scale data are widely used, that is, concept forgetting.\n2. The authors formally define the concept forgetting and empirically show that existing end-to-end finetuning methods except L2SP severely suffers from the concept forgetting.\n3. The paper proposes a simple yet effective regularizer, LDIFS, to alleviate the concept forgetting.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, the authors conducted extensive experiments, which effectively highlight the contributions of the paper. Please address the weaknesses mentioned above.\n\nMinor question: Does \"LPIPS\" in the provided code refer to \"LDIFS\"?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YitVNpJQXH", "reviewer": "Reviewer_wCd4", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nThe experiment is comprehensive in terms of downstream tasks.\n\n\nWeaknesses:\n\n1. The paper only considers one foundation model, i.e., CLIP ViT-B/32. It is unclear whether the conclusion generalizes to larger models like CLIP ViT-L. \n\n2. The reason why preserving feature space is better than preserving parameter space is not well explored.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper shows a concept forgetting phenomenon in the fine-tuning process of a foundation model, where fine-tuning on one task leads to performance deterioration on other tasks. The paper presents extensive experiments to demonstrate the concept forgetting issue on 9 datasets. A feature preserving regularization (LDIFS) is proposed to fix this issue and shown to be effective on multiple datasets compared with several baselines.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add a larger foundation model in the empirical study\n\n2. Explore the reason why LDIFS is better than parameter constraint methods. \n\n3. Explain the effect of training sample size in a downstream task to the effectiveness of the proposed approach.", "broader_impact_concerns": "Not applicable.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: All (but one) reviewers agree that the paper should be accepted. This reviewer's main concern is that the proposed fine-tuning setting is not fully realistic. Still, the reviewer agrees that the "Claims And Evidence" and "Audience" criteria are met.
While I agree this is a small concern, the evaluation and suggested solution are of high interest to the community (this is also a question more of impact, which is not part of the acceptance decision).

In fact, I believe the paper should be considered for Featured Certification, given the breadth of the experiments and potential for interest in the community. The proposed solution is easy to realize and has the potential to show wide applicability, given this is a fundamental problem.

For the minor revision, I would like to see:
Required:
1. Include the appendix as part of the main pdf
2. Include Split-ImageNet-R somewhere (if it is not added yet)
3. Include any aspects of the author's response not integrated so far

Optional:
1. I think the paper would be more impactful if the experiments provided in the author's response / revision of the appendix would not be an afterthought in the appendix but were properly integrated into the main paper, especially  Appendix C.5. I think this would give the paper a chance at the journal to ICLR conference track.
Decision: Accept (poster)