Contrastive Adapters for Foundation Model Group Robustness
Abstract
While large pretrained foundation models (FMs) have shown remarkable zero-shot classification robustness to dataset-level distribution shifts, their robustness to group shifts is relatively underexplored.
We study this problem, and first find that popular FMs such as CLIP may not be robust to various group shifts.
On prior robustness benchmarks, they achieve up to an 80.7 percentage point (pp) gap between average and worst-group accuracy.
Unfortunately, current methods to improve robustness require retraining, which can be prohibitively expensive for large FMs.
We find existing ways to efficiently improve large model inference, e.g , by training adapters (lightweight
MLPs) on top of FM embeddings, can also hurt group robustness compared to zero-shot.
We thus propose a first adapter training method designed to improve FM robustness to group shifts.
While prior work only trains adapters with class labels, we add a contrastive objective to explicitly learn similar embeddings for initially dissimilar FM embeddings.
Across the same benchmarks, contrastive adapting effectively and efficiently improves group robustness, raising worst-group accuracy by 16.0 to 56.0 pp over zero-shot without any FM finetuning.
Beyond FM robustness, contrastive adapting achieves near-state-of-the-art robustness on Waterbirds and CelebA, while only training 1% of other methods’ model parameters.
1. Introduction
Foundation models (FMs)—large pretrained models trained on massive datasets—offer an exciting new deep learning paradigm.
Recent works show that with no finetuning, FMs can generalize to various tasks (Brown et al., 2020;
Radford et al., 2021) with impressive robustness to certain distribution shifts (Kumar et al., 2022; Wortsman et al., 2021).
However, an underexplored question is how robust this no finetuning, or zero-shot , paradigm is to “group shifts,” distribution shifts between subpopulations or meaningful groups in data.
Prior works have established that group robustness – i.e performing well on all groups—is a fundamental and real-world challenge for modern deep learning (Beery et al., 2018;
Buolamwini & Gebru, 2018; Oakden-Rayner et al., 2020; Koh et al., 2021).
Yet most prior foundation model evaluations focus on overall or average performance; few works consider their accuracy across groups.
In this work, we thus study FM group robustness.
We first motivate this problem by showing that FMs can have poor zero-shot group robustness.
Evaluating 9 FMs across 7 robustness benchmarks, we find up to an 80.7 percentage point (pp) gap between their average and worst group accuracy.
We therefore aim to improve FM group robustness.
This poses several challenges and open questions.
First, current robustness methods often retrain at least one model (Sagawa et al., 2019;
Nam et al., 2020; Creager et al., 2021; Liu et al., 2021;
Ahmed et al., 2021; Zhang et al., 2022).
This can be prohibitively expensive for FMs due to their size and scale, and it is unclear if we can improve FM robustness without any retraining or finetuning.
Second, many practitioners may only access FM outputs or embeddings ( e.g , via APIs 1 ), using zero-shot classification for their downstream tasks.
Ideal robustness solutions would then also only require FM embeddings.
However, if these same embeddings lead to poor zero-shot robustness, then it is unclear if they encode the information needed to classify all groups correctly.
Motivated by these challenges and questions, we study effective and efficient solutions for better FM group robustness.
As a baseline, we first find that while efficient methods to improve FM inference– e.g , training linear probes (Radford et al., 2021; Kumar et al., 2022) and adapters (Houlsby et al., 2019;
Gao et al., 2021) on top of FM embeddings–can improve robustness over zero-shot ( e.g , reducing the accuracy gap by 20.4 pp on the Waterbirds dataset (Welinder et al., 2010)), they fail to do so consistently, and can hurt robustness (increasing the gap by up to 74.9 pp on the CelebA dataset (Liu et al., 2015)).
To improve robustness, we note that while poor zero-shot robustness occurs if FMs embed
[Figure]
f(u) u Adapter f vvf(u) · vŷ =(a) vv 0 f(u) (b) (c) f(u) pppnnn 2 Cross-entropy (ERM) Adapter
[Figure]
same-class samples in different groups “far apart”, we find instances where standard adapters fail to close this distance.
To consistently improve group robustness over zero-shot, we propose contrastive adapting , a simple adapter training method that places greater emphasis on bringing these initially “far apart” points together.
For each task, we first compute FM embeddings for each training sample and class.
We then train adapters—small bottleneck MLPs—on these embeddings.
Like prior work (Gao et al., 2021), these adapters take sample embeddings as inputs, and output transformed embeddings with greater cosine similarity to their groundtruth class embeddings.
However, the key difference is that contrastive adapting also applies contrastive learning over other sample embeddings.
We provide a way to “pull together” far apart sample embeddings in the same class, and “push apart” nearby sample embeddings in different classes.
In our experiments, we validate that contrastive adapting effectively and efficiently improves FM group robustness.
First, across all 9 robustness benchmarks, we find contrastive adapting consistently improves worst-group accuracy over zero-shot (by 16.0 to 56.0 pp), using no training group labels and adapters with 0.1% to 0.3% of the original FM parameters.
Then, on the popular Waterbirds and
CelebA robustness datasets, we find contrastive adapting can substantially outperform other methods that only use
FM embeddings.
Finally we find contrastive adapting can enable effective and efficient group robustness in general.
On CelebA, we achieve +0.2 pp worst-group accuracy over the prior state-of-the-art with only 1.0% of its parameters.
2. Problem 2.1. Preliminaries: group robustness and task setup
For setup, we follow prior work (Sagawa et al., 2019).
For some task, we have N samples { ( x i , y i , g i ) } Ni =1 , with sample inputs x i ∈ X , class labels y i ∈ Y , and group labels g
i ∈
G .
Let C = |Y| be the number of classes.
g i indicates each sample’s group, but we do not see group labels during training.
Distribution shifts may occur between same-class samples in different groups.
Every sample ( x i , y i , g i ) is drawn from some joint distribution P .
Let P g be the specific distribution conditioned on g for any g ∈ G .
For classification loss ℓ : Y × Y (cid:55)→ R and classifier f θ : X (cid:55)→ Y , we want f θ to be accurate, achieving low average error:
[Equation]
and group robust , achieving a small gap between average error and worst-group error (implying low worst-group error):
[Equation]
Unlike out-of-distribution (OOD) evaluation, we observe each group in all data splits.
However, standard training via empirical risk minimization (ERM) can lead to poor group robustness due to imbalanced groups (Shimodaira, 2000).
Here, FMs are not trained on the training data, but we show their zero-shot classification may still not be group robust.
2.2. Empirical findings of poor FM group robustness
We now demonstrate the group robustness problem with foundation models.
We first briefly describe zero-shot classification and baseline methods (linear probes, adapters) to improve inference over the former.
We then show that popular FMs such as CLIP (Radford et al., 2021) may not be group robust on various robustness benchmarks from prior work.
We finally use two datasets, Waterbirds (Welinder et al., 2010; Sagawa et al., 2019) and CelebA (Liu et al., 2015), to present two outcomes where baselines can help and hurt robustness.
Additional benchmark details and results for all 9 FMs evaluated are deferred to Appendix A.
Baseline methods.
To evaluate foundation model group robustness, we consider the following baseline methods:
• Zero-shot classification (Radford et al., 2021):
Assuming C classes, we first use a FM to compute sample embeddings for each test sample and C total class embeddings.
We then use the nearest class embedding (via cosine similarity) to each sample embedding for class prediction.
For class embeddings, we convert each class to a text prompt, e.g , “this is a [class name] ”, and use the FM text encoder output of the tokenized prompt (c.f. Fig. 1).
• Linear Probe (Radford et al., 2021; Wortsman et al., 2021): We train a linear classifier on top of training data sample embeddings to predict their class labels.
We then use the linear probe to classify test sample embeddings.
• Adapter (Gao et al., 2021; Rebuffi et al., 2017):
We train a single 2-layer bottleneck multilayer perception (MLP) to output transformed sample embeddings most similar to their ground-truth class embeddings (c.f. Fig. 1).
Specifically, with adapter f θ , sample embedding u , ground-truth class embedding v , temperature τ , and l2-norm operator ˆ · , we update adapter weights θ with a cross-entropy loss
[Equation]
At test time, we compute a sample embedding for each test sample, and use these “adapted” embeddings instead of the FM embeddings in the zero-shot procedure above.
For class embeddings, we tried several templates, choosing by val.
set worst-group acc.
App C.6 lists the templates.
Zero-shot classification results.
Fig.
2 charts the average and worst-group accuracies achieved via zero-shot classification with a CLIP ResNet-50 (RN-50) FM (Radford et al., 2021) on the 7 image benchmarks.
We find evidence of poor group robustness (with accuracy gaps up to 80.7 pp).
The larger gaps suggest that while FMs may learn correlations that apply to many unseen samples on average ( e.g , classifying 80+% of samples correctly on Waterbirds and BREEDS datasets (Santurkar et al., 2020)), such correlations may not hold for all groups, leading to poor worst-group accuracy (only 49.8% and 6% for Waterbirds and BREEDS datasets).
Efficient baseline results.
In Table 1, we report worstgroup and average accuracy with all baseline methods on the popular Waterbirds and CelebA benchmarks.
This presents two different outcomes.
Perhaps surprisingly, ERM-trained adapters are sufficient to improve robustness significantly on Waterbirds (+24.4 pp worst-group accuracy, -20.4 pp accuracy gap).
However, on CelebA, both baseline methods result in poorer group robustness than zero-shot.
Motivation for improving robustness over baselines
Towards improving robustness consistently, we expand on the FM group robustness problem with possible limitations of ERM adapter training.
Note that if zero-shot classification for a class is accurate on average but not group robust, then in the FM embedding space there exist groups embedded “far apart” despite being in the same class.
One view of
ERM adapter training is that it trains adapters to learn an embedding space that brings these groups together.
We can interpret Eq.
3 as a contrastive loss (Oord et al., 2018;
Chen et al., 2020) where u is an anchor, v is a positive, and the other C − 1 class embeddings are negatives.
While this works for Waterbirds, only using class embeddings as
Waterbirds
CelebA
[Table]
positives and negatives may not always bring desired points together.
We verify this in Fig.
3 by computing the mean pairwise cosine similarity of intra-class sample embeddings in the same class but different groups.
On adapter-trained and pretrained embeddings from multiple CLIP models, poorer robustness tracks lower intra-class cosine similarity.
3. Method: Contrastive Adapting
To improve robustness, we thus aim to more explicitly bring far away samples together.
For an anchor sample, instead of using only 1 class embedding positive and C − 1 negatives, we add positives with sample embeddings in the same class but far away from the anchor ( e.g , likely in different groups).
We add negatives with sample embeddings nearest to the anchor but in different classes (as Ge et al.
; Zhang et al.
(2022) show hard negatives benefit contrastive learning for robustness).
As dataset size is often much larger than number of classes, contrastive adapting is also supported by Khosla et al.
; Robinson et al.
(2021) that show more positives and negatives benefit contrastive learning.
Our method is simple to implement with three components:
[Table]
(1) Foundation model embedding and prediction : We compute FM embeddings over labeled training data.
For (2), we also compute zero-shot predictions on this data.
(2) Contrastive sampling : For each class, we identify “anchor” sample embeddings that zero-shot predicts incorrectly.
For each anchor, we sample P “positive” sample embeddings that zero-shot classifies correctly (as a heuristic for finding “far apart” samples), and M hard “negative” sample embeddings by computing the nearest neighbors to the anchors in different classes by cosine similarity.
(3) Training objective : We update adapters with a contrastive sample loss (Khosla et al., 2020) on the 1 + P + M sample embeddings.
To ensure that the adapter learns to keep sample embeddings close to their ground-truth class embeddings, we also use a cross-entropy loss over random minibatches of sample embeddings (c.f. Eq. 3).
4. Experiments
We aim to validate that contrastive adapting improves and enables efficient and effective group robustness with FM embeddings.
First, across image benchmarks, contrastive adapting consistently improves group robustness over zeroshot.
Next, on Waterbirds and CelebA, contrastive adapting is more effective than other strategies to improve inference with only FM embeddings.
Finally, on these datasets, contrastive adapting enables state-of-the-art robustness with greater parameter efficiency.
We select models based on val.
set worst-group accuracy as in prior work (Koh et al., 2021).
App C.2 contains method details and hyperparameters.
[Table] [Table]
Consistent robustness improvements over zero-shot.
In
Fig 4, we find that unlike ERM adapters, contrastive adapting consistently improves group robustness on the 7 image robustness benchmarks and CLIP RN-50 over zero-shot, achieving 16.0 to 56.0 pp higher worst-group accuracy.
Robustness improvements over efficient FM methods.
In
Table 3, we find contrastive adapting improves group robustness over prior baselines and other strategies to improve
FM inference with training sample FM embeddings: WiSEFT (Wortsman et al., 2021), which ensembles linear probe and zero-shot weights; DFR (Kirichenko et al., 2022) which trains linear probes on resampled data; and group prompt zero-shot, where we use group-informed prompts for class embeddings.
On average, contrastive adapting improves worst-group accuracy over the next best method by 16.4 pp.
Efficiency improvements over recent robust methods.
In
Table 2, we find that contrastive adapting with CLIP RNembeddings achieves comparable worst-group accuracy to recent state-of-the-art robustness methods on Waterbirds and CelebA, despite only training 1% of their parameters.
Notably, compared to the state-of-the-art Spread Spurious
Attribute (SSA) (Nam et al., 2022), contrastive adapting achieves +0.2 pp worst-group accuracy on CelebA.
5. Conclusion
We find that FM zero-shot classification may not be grouprobust, and present a simple first-step approach to significantly improve robustness without any finetuning.
A. Expanded zero-shot evaluation for group robustness
In this section, we expand on the zero-shot evaluation of various foundation models on group robustness benchmarks discussed in Section 2.
We first describe the datasets and models used in Appendix A.1.
We then include results in
Appendix A.2.
We also describe and evaluate on two additional text group robustness datasets with two natural language foundation models.
We find consistent trends of poor group robustness with zero-shot classification, marked by poor worst-group accuracy and large gaps between average and worst-group accuracy.
A.1. Additional details on robustness datasets and foundation models
Datasets.
To benchmark zero-shot group robustness, we use a diverse set of datasets with group shifts from prior robustness literature.
We describe them below and include details on size of groups and type of group shift in Table 4:
• Waterbirds (Welinder et al., 2010; Sagawa et al., 2019).
We classify images by bird type.
Each class ∈
{ waterbird , landbird } carries two groups: birds on water backgrounds, and birds on land backgrounds.
• CelebA
(Liu et al., 2015; Sagawa et al., 2019).
We classify images by celebrity hair color.
Each class ∈
{ not blond , blond } carries two groups: celebrities labeled as male, and celebrities labeled as female.
• BREEDS ( Living-17 , Nonliving-26 ) (Santurkar et al., 2020).
For the Living-17 and Nonliving- datasets in the BREEDS benchmark sourced from ImageNet (Santurkar et al., 2020), we classify images by one of several categories.
Each class is a coarse category consisting of multiple fine-grained groups.
Groups in the same class may be visually distinct ( e .g the ape class
includes
images of gibbons and gorillas). While the original benchmark evaluates how classifiers trained on seen source groups generalize to unseen target groups, we adapt the datasets for our group robustness setting by adding 5% of the images in each target group
to the
source groups
and evaluating
worst-group accuracy over all source and target groups. • CIFAR-10.001 , CIFAR-10.02 (Krizhevsky, 2009; Recht et al., 2018;
Lu et al., 2020).
We classify images by one ofcategories.
We combine CIFAR-10 (Krizhevsky, 2009) and either CIFAR-10.1 (Recht et al., 2018) or CIFAR-10.2 (Lu et al., 2020), which are collected from different sources.
The new datasets’ classes carry two groups determined by the source dataset.
• FMoW-WILDS (Christie et al., 2018; Koh et al., 2021).
We classify satellite images into one of 62 building or land-use categories ( e.g , airport , zoo ).
Each images belongs to one of five groups based on continental region.
To test group robustness, we compare the accuracies over all samples in each group as in the WILDS benchmark (Koh et al., 2021).
We also evaluate only over test images from the same time period as training images (the “IID” split in the original WILDS benchmark (Koh et al., 2021)).
• CivilComments-WILDS (Borkan et al., 2019; Koh et al., 2021).
We classify if a text comment is toxic or not.
Samples are organized into 8 groups based on mention of a demographic identity ( e.g , “female”, “LGBTQ”).
• Amazon-WILDS
(Ni et al., 2019; Koh et al., 2021).
We classify if an online text review is positive or negative.
Reviews are organized into different groups based on the product category ( e.g , books , electronics ).
We adapt this dataset from the official Amazon-WILDS split by using the category subpopulation split.
We also map the original class labels, which are star-ratings from 1 to 5, to positive or negative reviews by discarding samples with a 3-star rating, and re-labeling 1- and 2-star ratings as negative and 4- and 5-start ratings as positive .
Foundation models.
For image datasets, we evaluate pretrained CLIP (Radford et al., 2021) and CLOOB (F¨urst et al., 2021) vision-language models using publicly available weights 23 .
We evaluate
[Table]
Dataset
[Table]
Waterbirds
[Table]
CelebA
[Table]
BREEDS Living-
[Table]
BREEDS Nonliving-
[Table]
CIFAR-10.
Data
[Table]
CIFAR-10.
Data
[Table]
FMoW-WILDS
[Table]
No
Amazon-WILDS
[Table]
No
CivilComments-WILDS
Confounder
[Table]
Yes
A.2. Group robustness results
In Figure 5, we chart worst-group and average accuracies achieved by various zero-shot foundation models across the group robustness datasets.
Larger gaps between accuracies, i.e high average accuracy yet low worst-group accuracy, indicate poor group robustness.
In aggregate, on all datasets except FMoW-WILDS and Amazon-WILDS, we observe a shared pattern of noticeable gaps between average and worst-group accuracy, suggesting that zero-shot classification with popular foundation models may not be group robust.
We perform zero-shot classification as described in Section 2.
As recommended by
Radford et al.
(2021), for each dataset we consider several prompt templates.
We engineer prompts by using the single best template based on validation worst-group accuracy.
In Appendix C.6 we include a full list of prompts used.
B. Contrastive adapter implementation details
We provide further details on the adapter architecture and training sampling.
B.1. Adapter architecture
[Figure]
Similar to prior works (Houlsby et al., 2019; Gao et al., 2021), the adapters we use are bottleneck 2-layer multilayer perceptrons (MLPs).
We set the input dimension and output dimensions as the same as the pretrained foundation model embedding dimension, and pick a smaller dimension for the hidden layer (frequently 128, although this was chosen as a heuristic and not tuned).
We also experimented with using a single residual connection (He et al., 2016) and batch normalization layer (Ioffe & Szegedy, 2015) between the input and output layers, but only found the latter to be helpful.
Pytorch-like pseudocode is given below.
The adapter is visualized in Figure 6.
def __init__(self, input_dim
hidden_dim):
super().__init__()
self.arch = nn.
Sequential( nn.Linear(input_dim, hidden_dim), nn.BatchNorm1d(hidden_dim), nn.ReLU(), nn.
Linear(hidden_dim, input_dim) ) def __forward__(self, x): return self.arch(x) B.2. Adapter training sampling
Recall that we train the adapter with both a supervised contrastive loss over specifically sampled “contrastive batches”, and a cross-entropy loss over resampled batches, both over the fixed pretrained foundation model embeddings.
We use the foundation model’s zero-shot classification predictions to guide sampling for both.
We outline the algorithms for sampling training batches in Algorithm 1 and Algorithm 2.
We then train an adapter by applying the sample-wise supervised contrastive loss (sample-wise SupCon)
(Khosla et al., 2020) and the cross-entropy loss (Eq. 3) over these batches in Algorithm 3.
M embeddings
= { u i }
C. Additional experimental details C.1. Main results comparison methods
In Table 3, we compared contrastive adapting with several other approaches designed to improve downstream transfer in related settings, while similarly only requiring pretrained model embeddings.
We describe them in more detail below:
• Weight space ensembling (WiSE-FT) (Wortsman et al., 2021), which first trains a linear classifier with standard ERM, and then ensembles the classifier outputs with the initial zero-shot predictions.
While proposed for both training linear classifiers and finetuning the original weights of a foundation model, we focus on the linear classifier version for fair comparison in our setting.
• Deep feature reweighting (DFR) (Kirichenko et al., 2022), which first trains a linear probe on embeddings computed from a pretrained model over group-balanced data.
As we do not assume training group labels, we first infer groups using zero-shot classification with foundation model embeddings.
As in prior work (Liu et al., 2021; Zhang et al., 2022), we treat the incorrect and correctly classified samples as proxies for different groups.
Finally, assuming we have validation group labels, we know what groups could plausibly be in our test data.
We thus also compare against group-informed prompting , which performs zero-shot classification using prompts with group information ( e.g , “a waterbird on a land background”).
C.2. Model selection and hyperparameters
For each dataset and method, we use the following hyperparameters.
As in prior group robustness work (Koh et al., 2021), we select the best model and hyperparameters based on early stopping that achieves highest worst-group validation accuracy.
For all methods and datasets, we train both linear probes and adapters with SGD, and sweep over learning rate ∈ { 1e-3, 1e-4, 1e-5 } and weight decay ∈ { 5e-5, 1e-5, 5e-4 } .
For adapter classification, we used the default temperature used for zero-shot classification in CLIP (Radford et al., 2021).
We did not tune the contrastive temperature.
Unless noted, we ran all numbers.
We list hyperparameters for linear probes (Table 5), adapters (Table 6, both ERM and contrastive), and contrastive-specific hyperparameters (Table 7).
We discuss method-specific hyperparameters:
• Contrastive adapting requires selecting three additional hyperparameters: the number of positives and negatives, and the number of nearest neighbors to sample negatives from.
For these we swept over the following combinations of (number positives, number negatives, number neighbors): (2048, 2048, 2146), (2048, 2048, 4096), (512, 512, 1024).
• Weight-space ensembling (WiSE-FT): WiSE-FT requires picking a value α ∈ [0 , 1] to compute a weighted combination of the zero-shot classifier parameters and the trained linear probe parameters.
We sweep over intervals of size 0 .
1 , i.e α ∈ { 0 . 0 , 0 .
[Table]
Waterbirds
[Table] CelebA [Table]
[Table] BREEDS Living1e-
[Table]
BREEDS Nonliving1e- 5e0.
CIFAR-10.
1e5e- 0.
CIFAR-10.
1e5e-
[Table]
FMoW-WILDS [Table] - 5e0.
Amazon-WILDS 1e [Table] 5e
[Table]
CivilComments-WILDS 1e5e- 0.
C.3. Data splits
We use the same train, validation, and test splits for Waterbirds, CelebA, FMoW-WILDS, Amazon-WILDS, and
CivilComments-WILDS as in prior work.
For BREEDS and CIFAR datasets that we adapt for our problem setting, we construct test splits by combining official test splits from the original benchmarks.
We then create training and validation splits by combining the rest of the data from these benchmarks, and randomly splitting this into 80% training data and 20% validation data.
No original test data is seen during training on our splits.
[Table]
[Table]
C.4. Additional dataset assets details and discussion
Dataset licenses.
To curate CIFAR-10.0001 we use the CIFAR-10.1 dataset, which is distributed under the MIT License.
The FMoW-WILDS dataset is distributed under the FMoW Challenge Public License 6 .
The CivilComments-WILDS dataset is distributed under CC0 1.0.
The Amazon-WILDS dataset does not have a license, but is requested to be used for research purposes only (Koh et al., 2021).
We were not able to find explicit license information for CIFAR-10.2, Waterbirds, CelebA, or the BREEDS datasets.
We note that the BREEDS datasets are sourced from ImageNet, which is distributed under the
BSD 3-Clause License, and set up with code from the MadryLab robustness GitHub repository 7 , which is distributed under a MIT license.
The authors of the CelebA dataset provide a list of agreements 8 , including that the dataset is used only for non-commercial research purposes.
Existing assets personally identifiable information and offensive content.
The CelebA dataset consists of images of celebrity faces, which are personally identifiable.
The dataset is also categorized by male and female identification at the time of curation, which may be outdated.
The CivilComments-WILDS contains text samples flagged as toxic by toxicity classifiers (Koh et al., 2021), which contain potentially offensive content.
Both datasets are existing assets, and both personal identifiability for CelebA and offensive content for CivilComments-WILDS can be checked by inspecting the original data inputs (images and text comments).
C.5. Compute and resources
All experiments were run on a machine with 14 CPU cores and a single NVIDIA Tesla P100 GPU.
For training a contrastive adapter on top of CLIP ResNet-50 Waterbirds embeddings, this took approximately 30 minutes to run 100 epochs.
Other than the numbers reported from their original publications in Table 2, we report all numbers from running experiments on the same machine.
[Table]
Prompt template / example of
[Table]
CLIP “This is a picture of a [class name] .”
CLOOB “a [class name] ”
CelebA
CLIP “A photo of a celebrity with blond hair.”
CLOOB “A photo of a celebrity with blond hair.”
BREEDS Living
[Table] CLIP “This is a picture of a [class name] .” CLOOB “This is a picture of a [class name] .”
BREEDS NonlivingCLIP “A photo of a [class name] .”
CLOOB “a [class name] ”
CIFAR-10.
CLIP “a [class name] ”
CLOOB “a [class name] ”
CIFAR-10.
CLIP “a [class name] ”
CLOOB “a [class name] ”
FMoW-WILDS
CLIP “satellite view of the [class name] ”
CLOOB “aerial view of an [class name] ”
Amazon-WILDS
GPT-Neo “Negative”
CivilComments-WILDS
GPT-Neo “Not toxic”
C.6. Class prompt templates
In Table 8, we list the templates used to generate class prompts for each dataset.
As a reminder, for each provided class name in a dataset, we create a prompt by inserting the class name into the prompt template.
We then encode this prompt with a foundation model text encoder to get class embeddings.
D. Additional related work discussion
We provide additional discussion of related work and connections to our work below.
Zero-shot classification with foundation models.
Our work builds on a growing literature on applying foundation models, large pretrained models that can be applied to various downstream tasks.
These models demonstrate exciting promise in their ability to achieve accurate downstream transfer without any additional finetuning (Brown et al., 2020; Radford et al., 2021; Bommasani et al., 2021).
In particular we consider the zero-shot capabilities of pretrained vision-language foundation models.
These models, such as CLIP (Radford et al., 2021), ALIGN (Jia et al., 2021), and CLOOB (F¨urst et al., 2021) are trained on massive amounts of naturally paired image-text data, e.g , Internet images and their corresponding captions.
Consisting of an image encoder (usually a ResNet or Vision Transformer) and a text encoder (usually a Transformer), such foundation models are commonly trained to learn a shared image-text embedding space where embeddings of images are most similar to embeddings of their corresponding caption text.
While these objectives have been shown to lead to powerful representations (Zhang et al., 2020; Desai & Johnson, 2021), a crucial element for successful zero-shot classification is training data scale (Radford et al., 2021).
However, added scale can also be a double-edged sword; when zero-shot classification still makes undesirable mistakes, standard ways to correct for these mistakes via retraining can become prohibitively expensive.
We study one such motivating instance via group robustness, and provide a first-step solution towards improving group robustness efficiently.
Improving foundation model inference efficiently.
Prior works aim to improve foundation model downstream performance, without having to finetune or update original model weights.
Prompt tuning optimizes the inputs of a FM while keeping the original model weights frozen.
Optimizing either text (Li & Liang, 2021; Zhou et al., 2021; 2022; Levine et al., 2022) or image (Bahng et al., 2022;
Yao et al., 2021) inputs can improve a frozen foundation model’s downstream task accuracy.
However, doing so can require multiple passes through the foundation model, which may become expensive in certain situations ( e.g , interacting with a foundation model via a commercial API).
Another paradigm adds small trainable parameters to the original model, either within its layers or on top of its embeddings.
These include linear probes (linear classifiers) (Radford et al., 2021) and adapters (small bottleneck MLPs) (Houlsby et al., 2019; Rebuffi et al., 2017; Pfeiffer et al., 2021; 2020).
Recently, Kumar et al.
; Wortsman et al.
(2021) propose methods that use linear probes to improve robustness to dataset-level out-of-distribution shifts.
Gao et al.
(2021) propose to train single adapters on top of FM embeddings to improve average downstream task accuracy.
We focus on group shifts that occur within a dataset.
We also show that the latter can hurt group robustness, and propose alternatives to consistently improve group robustness.
Robustness of foundation models.
Prior works have studied the robustness of foundation model inference to natural distribution shifts.
Radford et al.
(2021) show that zero-shot CLIP models can be more robust to out-of-distribution (OOD) shifts than prior ImageNet-trained models, measured via better generalization to various dataset-level distribution shifts on
ImageNet classes.
However, they also show that finetuning, or updating the original weights, of CLIP models on ImageNet can reduce this OOD robustness.
Kumar et al.
; Wortsman et al.
(2021) thus propose finetuning methods that improve downstream in-distribution accuracy while maintaining out-of-distribution robustness.
Kumar et al.
(2022) specifically study the trade-off between linear probing and finetuning, finding that finetuning on downstream data can improve generalization on in-distribution data over linear probing but more substantially hurt performance OOD data than linear probing.
They show theoretically and empirically that a two-step strategy of first linear probing then full fine-tuning can combine the performance boosts of both.
Wortsman et al.
(2021) focus on the OOD trade-off presented by Radford et al.
(2021) between a finetuned foundation model and its pretrained zero-shot weights.
They propose weight-space ensembling (WiSE-FT), which computes a weighted average of the finetuned and pretraiend foundation model parameters, and show that the resulting averaged parameters can in some instances achieve higher performance on both data distributions that the model was finetuned on and unseen OOD data than the initial finetuned and zero-shot or pretrained models.
They show this effect with both full finetuning and training a linear probe.
Unlike these works, we focus on foundation model robustness to group shifts that occur within a dataset.
We also compare against the linear probe version of WiSE-FT, and find that training adapters can be advantageous for achieving higher group robustness on various datasets.
Recently, other works also study foundation model learned spurious correlations and biases.
Singla et al.
(2022) show how various models (including CLIP models) may rely on spurious artifacts to classify ImageNet images.
Berg et al.
(2022) aim to debias CLIP image embeddings of human faces using extra metadata (textual concepts or attributes) that the embeddings should ignore.
Our evaluation is complementary, noting poor group robustness across multiple types of data sources (objects, animals, human faces, text).
We also provide a method that works without additional training metadata.
Improving group robustness of deep learning models.
Improving the group robustness of deep learning models is a common deep learning challenge, where models may learn biases during training that lead to poor performance on certain groups.
This is a widespread issue presented in contexts ranging from algorithmic fairness to healthcare diagnosis (Blodgett et al., 2016; Hashimoto et al., 2018; Buolamwini & Gebru, 2018).
Several methods exist to improve group robustness.
We compare against several recent approaches in Section 4.
While one effective strategy to improve group robustness is to upweight the error of worst-performing group during training (Sagawa et al., 2019; 2020), training group labels may be impractical to obtain in practice (Sohoni et al., 2020; Oakden-Rayner et al., 2020).
We thus consider robustness approaches which aim to work without training group labels.
Several approaches involve training two models; one model is first trained with standard ERM to help infer groups, and another trained with a robust objective using these inferred groups.
Just
Train Twice (JTT) (Liu et al., 2021) treats samples that the first model misclassifies as inferred minority group samples to upweight.
JTT then upweights these samples by a hyperparameter factor, and trains a second model with ERM on this upsampled data.
Environment Inference for Invariant Learning (EIIL) (Creager et al., 2021) infers groups by assigning samples to group under which the ERM model maximally violates an Invariant Risk Minimization (Arjovsky et al., 2019) principle.
It then trains a robust model with Group DRO (Sagawa et al., 2019) using the inferred groups, which dynamically upweights the worst-performing groups during training.
Correct-N-Contrast (CNC) (Zhang et al., 2022) instead identifies samples with the same class labels but different ERM model predictions, and trains a robust model by using a contrastive loss to learn similar representations between these samples.
Spread Spurious Attribute (SSA) (Nam et al., 2022) specifically trains the first model to predict groups using a small set of group labels, before using Group DRO to train a robust model.
Contrastive Input Morphing (CIM) (Taghanaki et al., 2021) trains a network to transform the input features of an image to better present class-specific information shared across groups.
Idrissi et al.
(2021) suggest that simply changing the training data by subsampling large classes (SUBY) or balancing the class sampling probabilities (RWY), then training a model with
ERM, can also improve group robustness.
E. Additional experimental results E.1. Extended main results
In Table 11 and Table 12 we report group robustness results evaluating all methods discussed in Section 4 on all group robustness benchmarks.
Table 11 contains results for image datasets, using CLIP-RN50 embeddings.
Table 12 contains results for text datasets, using GPT-Neo 1.3B embeddings.
As in Table 3, we report the worst-group and average accuracies, along with their gap.
Higher worst-group accuracy and smaller accuracy gap are indicative of better group robustness.
All results are computed over three random seeds, with mean and one standard deviation included (error bars deferred to here from the main paper).
Compared to alternative methods, contrastive adapting consistently improves group robustness over zero-shot classification, and obtains highest worst-group accuracy and smallest accuracy gap on datasets where training adapters with ERM fails.
On datasets where ERM-trained adapters achieve best group robustness, contrastive adapters are also competitive or closest to ERM-trained adapters among other robustness methods.
E.2. Contrastive adapter ablations
In this section, we ablate different training components of contrastive adapting.
We show that the presented combination leads to best worst-group accuracy on the Waterbirds dataset.
We also study how the number of positives and negatives used in contrastive sampling affects performance, and find that models do seem to benefit from a greater number of samples.
E.2.1. T RAINING COMPONENT ABLATIONS
We study the importance of the contrastive and cross-entropy components in contrastive adapting.
For evaluation, we use the Waterbirds dataset, and run ablations comparing adapters trained on top of CLIP embeddings with (i) no contrastive component, (ii) no cross-entropy component, or the default proposed approach.
We evaluate across five different CLIP models.
All other training procedures are kept consistent.
In Table 9, we report worst-group accuracies.
We find that both contrastive and cross-entropy components are necessary for best worst-group accuracy.
The contrastive objective leads to a substantial improvement over just the resampled cross-entropy loss (+17.9 pp on average).
However, we also note that without the cross-entropy objective to learn sample embeddings close to their ground-truth class embeddings, we observe high variance in classification accuracy.
We improve +26.9 pp on average using both objectives compared to contrastive alone.
Training Component Ablation
[Table]
No contrastive
[Table]
. E FFECT OF CONTRASTIVE BATCH SIZE
While one advantage of training adapters is that because we train on embeddings, the memory size of our data inputs during training is much smaller than the traditional alternative ( e.g , storing an tensorized image).
We can thus train with larger batch sizes.
Here we study how contrastive batch size, i.e how many positives and negatives we sample per anchor, affects worst-group accuracy.
On the Waterbirds and CelebA datasets and with CLIP RN-50 embeddings, we train a contrastive adapter with varying levels of positives and negatives.
For both datasets, the default is 2048 positives and 2048 negatives per batch.
We ablate these numbers with the following (positive, negative) combinations: (1, 1), (2, 2), (256, 256), (256, 512), (512, 256), (512, 512), (512, 1024), (1024, 512), (1024, 1024), (1024, 2048), (2048, 1024).
In Figure 7, we plot the effect of smaller batch sizes on worst-group accuracy.
We find that larger batch sizes weakly correspond to higher worst-group accuracy on both Waterbirds and CelebA.
However, perhaps surprisingly, we still maintain a substantial improvement over zero-shot classification with just a single positive and negative per anchor.
E.3. Comparison to TIP-adapter and training sample nearest-neighbors lookup
On representative benchmarks, we perform further comparison to the nearest-neighbor look-up approach employed by TIP
Adapter (Zhang et al., 2021).
Instead of learning transformed representations of pretrained embeddings, another approach to better classify a test sample is to use the class of its nearest training sample.
Under the assumption that the training and test data are sampled from the same broader distribution and share the same groups, then test samples in a given group should embed closest to training samples in the same group.
The training sample ground-truth class should then apply to the test sample.
TIP adapter operates accordingly, keeping a cache of training sample embeddings available at test-time.
One advantage is this allows for potentially more accurate classification without any training.
To test how well this idea fares for group robust classification, for each test sample we perform a look-up with all training samples, using cosine similarity to identify nearest neighbors.
In Table 10, we compare TIP-adapter with zero-shot classification and contrastive adapting on the Waterbirds, CelebA,
BREEDS Living-17, and CIFAR-10.02 group robustness benchmarks.
For all methods, we use CLIP RN-50 pretrained embeddings.
We find that TIP adapter improves worst-group accuracy over zero-shot classification on 3 out of 4 datasets, and notably achieves best worst-group accuracy on BREEDS Living-17 without training any additional parameters.
However, the improvements are more marginal on Waterbirds and CIFAR-10.02.
Contrastive adapting still achieves 30.4 pp higher worst-group accuracy over TIP adapter on average.
This may suggest that learning a nonlinear transformation of the pretrained embeddings can still be helpful for better “presenting” class-specific information to classify samples by. Waterbirds CelebA BREEDS LivingCIFAR-10.
[Table]
Gap
[Table]
Gap WG Avg Gap
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table]
[Table] [Table] [Table] [Table]
[Table]
[Table]
[Table]
[Table]
To provide additional perspective on how different embedding-only methods trade-off worst-group and average accuracy, we compare how these methods perform with respect to the accuracy trade-off traced out by weight-space ensembles.
Wortsman et al.
(2021) show an interesting phenomenon where simply taking a weighted average of a trained linear probe and the original foundation model (either over the weights, or the outputs) can result in a “pareto frontier” of accuracy metrics.
They specifically show that a weight-space ensemble can often achieve better OOD performance without sacrificing too much IID performance.
compared to a single linear probe.
In this context, we see how this trade-off occurs over average accuracy and worst-group accuracy across our representative set of group-robustness datasets.
We also evaluate how other approaches (ERM adapters, DFR (Kirichenko et al., 2022), contrastive adapters) fare along this trade-off.
In Figure 8, we plot the accuracies of these methods run on CLIP RN-50 embeddings.
We note several observations.
Weightspace ensembles (WiSE-FT) achieve the desired effect on certain datasets but not others.
On CelebA and CIFAR-10.02, we find that an ensemble can obtain a better worst-group accuracy versus average accuracy trade-off than either zero-shot classification or linear probes.
However, the single linear probe does at least as well as any ensemble in BREEDS Living-17, while the zero-shot classification does at least as well as any ensemble in Waterbirds.
We also find that among other methods, contrastive adapting is the only evaluated approach that consistently achieves higher worst-group accuracy than any weight-space ensemble.
While contrastive adapting places “above” the trade-off curve traced out by WiSE-FT on 3 out of 4 datasets (CelebA, BREEDS Living-17, and CIFAR-10.02), it tends to degrade average performance in favor of higher worst-group performance compared to other approaches.
Further work can improve how to raise worst-group performance without sacrificing any average performance.
F. Limitations and societal impact
Method limitations.
While in this work, we demonstrated that we can substantially improve the group robustness of foundation model classification without any finetuning of the original model, several limitations still exist.
First, this does not imply that we can get desirable performance in general without additional retraining.
To obtain high worst-group performance in general, we are upper-bounded by whether the pretrained embeddings do contain the information needed to classify all groups.
While our study suggests that in many cases they do carry this information—which can surprising given that the zero-shot classification with the same embeddings results in poor group robustness—in other situations the pretrained embeddings may lack this information.
For example, if downstream task data is very different in distribution from the pretraining data, then the pretrained foundation model embeddings may not be sufficient to work with.
While more efficient ways to improve robustness can democratize foundation model use, further finetuning may still be needed.
We also emphasize that our approach is a simple first-step method to improving the group robustness over existing baseline approaches.
This is motivated by our observation that foundation model zero-shot classification may not be group robust, and that we would like to both (i) improve performance of these models when we realize they fail in certain aspects, and (ii) do so efficiently, such that fixing their failures is not bound by who can conduct costly retraining procedures, and when they can do so.
We are excited for future work and expect further improvements as this thread of how to efficiently improve FM performance with limited access ( e.g , only pretrained embeddings) is further explored.
Societal impact and related limitations.
Finally, we note that it is important to carefully study the learned biases of foundation models, and to devise appropriate solutions evaluated outside of just computational metrics.
Due to their promise of widespread and effective downstream transfer, foundation models may have a particularly strong impact on various parts of society.
Individuals may get the sense that they can successfully apply these pretrained models to their desired downstream tasks “out-of-the-box”.
However, doing so also risks applying any learned biases of the model.
Our work raises this issue with respect to group robustness as motivation for our problem setting, and also notes that additional evaluation beyond average accuracy can shed light on the negative qualities of existing models (where zero-shot FM classification may perform very well on average, but very poorly on certain groups, c.f. Figure 5).
However we recognize the limitations of purely computational solutions to addressing group performance disparities in society.
We also recognize the need to better understand foundation models and their potential uses in broader socio-technical systems (Bommasani et al., 2021).
[Table]
[Table]