Contrastive Adapters for Foundation Model Group Robustness
Anonymous Author(s)
AffiliationAddress email
Abstract
While large pretrained foundation models (FMs) have shown remarkable zero-shotclassification robustness to dataset-level distribution shifts, their robustness to subpopulation or group shifts is relatively underexplored.
We study this problem, andfind that foundation models such as CLIP may not be robust to various group shifts.
Across 9 robustness benchmarks, zero-shot classification with their embeddings results in gaps of up to 80.7 percentage points (pp) between average and worst-groupaccuracy.
Unfortunately, existing methods to improve robustness require retraining,which can be prohibitively expensive on large foundation models.
We also findthat efficient ways to improve model inference ( e.g ., via adapters, lightweightnetworks that transform FM embeddings) do not consistently improve and cansometimes hurt group robustness compared to zero-shot.
We therefore develop anadapter training strategy to effectively and efficiently improve FM group robustness.
Our motivating observation is that while poor robustness results from groups inthe same class being embedded far apart in the foundation model “embeddingspace,” standard adapter training may not actually bring these points closer together.
We thus propose contrastive adapting, which contrastively trains adaptersto bring sample embeddings close to both their ground-truth class embeddings andsame-class sample embeddings.
Across the 9 robustness benchmarks, contrastiveadapting consistently improves group robustness, raising worst-group accuracy by8.5 to 56.0 pp over zero-shot.
Our approach is also efficient, doing so without any
FM finetuning and only a fixed set of FM embeddings.
On popular benchmarkssuch as Waterbirds and CelebA, this leads to worst-group accuracy comparable tostate-of-the-art methods, while only training ≤ 1% of the model parameters.
Introduction
Foundation models (FMs)—large pretrained models trained on massive datasets—offer an excitingnew paradigm for deep learning.
Recent works have shown that without any finetuning, foundationmodels can generalize well to various datasets [10, 32, 54, 62] and exhibit impressive robustness tocertain distribution shifts [38, 68].
Under this zero-shot paradigm, practitioners can avoid trainingtask-specific models, and instead use FM embeddings for efficient and effective inference.
However, an underexplored question is how robust this zero-shot inference is to “group shifts,”distribution shifts between subpopulations or meaningful groups in data.
Prior works have establishedthat group robustness — i.e . performing well on all groups—is a fundamental and real-world challengefor modern deep learning [4, 11, 36, 46, 50, 59, 64].
Yet most prior foundation model evaluationsfocus on overall or average performance [38, 54, 68]; few works consider FM accuracy across groups.
In this work, we thus study foundation model group robustness.
We motivate this problem by firstshowing that foundation models can have poor zero-shot group robustness.
Evaluating 11 foundation
Class Encoder
Sample Encoder f(u) u
Adapter f vvf(u) · vŷ =(a) vv 0 f(u) (b) (c) f(u) pppnnn 2 Cross-entropy Adapter
Contrastive Adapter
Adapter Classification models across 9 robustness benchmarks, we find they achieve up to an 80.7 percentage point (pp) gapbetween average and worst group accuracy, only classifying 6.0% of worst-group samples correctly.
We therefore aim to improve FM group robustness.
This poses several challenges and open questions.
First, while improving group robustness in machine learning is well-studied, existing robustnessmethods require retraining one (and often more than one) entire models [1, 15, 35, 42, 46, 58, 64, 65,71].
This can be prohibitively expensive for foundation models due to their size and scale, raisingthe question of whether we can make these models more robust without any retraining or finetuning.
Second, for zero-shot classification, many practitioners may also only access foundation modeloutputs or embeddings ( e.g ., via APIs 1 ).
To improve robustness, ideal solutions should only requirepretrained FM embeddings.
However, these same embeddings lead to poor zero-shot robustness,raising the question of if they even encode the information needed to classify all groups correctly.
Motivated by these challenges and questions, we study effective and efficient solutions for better FMgroup robustness.
As a baseline, we first find that while efficient methods to improve FM inference—such as training linear probes [38, 54] and adapters [21, 29] on top of FM embeddings—can improvegroup robustness over zero-shot (reducing the gap by up to 50.2 pp on representative benchmarks),they fail to do so consistently, and can hurt robustness.
They reduce worst-group accuracy by up to37.9 pp, and increase the accuracy gap by up to 74.9 pp.
To reason about this inconsistency, we notethat poor zero-shot robustness results when FMs embed same-class samples in different groups “farapart” in embedding space.
While adapter training achieves higher robustness than linear probing,we find settings where it still fails to close this distance, e.g ., if training data is group-imbalanced.
To then handle these scenarios and consistently improve group robustness over zero-shot, we proposecontrastive adapting , a simple adapter training method that places greater emphasis on bringingthese initially “far apart” points together.
For each task, we first use foundation models to computeembeddings for each training sample and class.
We then train adapters—small bottleneck MLPs—onthese embeddings.
Like prior work [21], these adapters take sample embeddings as inputs, andoutput transformed embeddings with greater cosine similarity to their ground-truth class embeddings.
However, the key difference is that contrastive adapting also applies a supervised contrastive lossover other sample embeddings.
Specifically, we provide a way to “pull together” far apart sampleembeddings in the same class, and “push apart” nearby sample embeddings in different classes.
In our experiments, we validate that contrastive adapting effectively and efficiently improves FMgroup robustness.
First, across all 9 robustness benchmarks, we find contrastive adapting consistentlyimproves worst-group accuracy over zero-shot (by 8.5 to 56.0 pp), using no training group labels andonly training MLPs with 0.1% to 0.3% of the original FM parameters.
Then, on a representative set ofbenchmarks with various group shifts and training data group sizes, we find contrastive adapting cansubstantially outperform prior adapter training strategies, and outperforms other approaches that onlyuse fixed FM embeddings (achieving up to 12.4 pp higher worst-group accuracy than the next bestmethod on average).
Finally, beyond just improving FM robustness, we find contrastive adapting alsoachieves effective and efficient group robust classification in general.
We achieve near state-of-the-art(SoTA) or SoTA worst-group accuracy on popular robustness benchmarks with only 1.0% of thetrainable parameters ( e.g ., improving 0.2 pp over the prior SoTA [47] on CelebA [43]).
In summary, we find that while FM zero-shot classification may not be group-robust, we cansignificantly improve robustness without any finetuning.
This suggests the information to classifygroups is frequently in their pretrained embeddings; we may just need proper methods to extract it.
Related Work
Our work builds on (i) methods to improve group robustness, and (ii) methods to improve foundationmodel inference without accessing or finetuning their original weights.
We briefly describe theseworks here, and include an expanded discussion in Appendix D.
Improving group robustness.
Many works aim to improve group robustness.
If training grouplabels are known, prior methods often balance group sizes during training, via sample balancing[16, 27, 30, 35], importance weighting [12, 61], or robust optimization [2, 58].
We do not assumetraining group labels.
With these assumptions, a common approach first trains a model with empiricalrisk minimization (ERM), before using this model’s predictions to infer groups.
Methods then train asecond robust model with sample balancing [42, 46] or robust optimization [15, 47, 64] using inferredgroup labels, or representation learning to learn similar representations for groups in the same class[71].
While effective at improving group robustness, these solutions require training one (and oftenmore than one) models.
This can make applying them to foundation models prohibitively expensive.
Improving foundation model inference efficiently.
Other prior works aim to improve foundationmodel downstream performance, without having to finetune or update original model weights.
Prompt tuning optimizes the inputs of a FM while keeping the original model weights frozen.
Optimizing either text [39, 40, 74, 75] or image [3, 69] inputs can improve a frozen foundation model’sdownstream task accuracy.
However, doing so can require multiple passes through the foundationmodel, which may become expensive in certain situations ( e.g ., interacting with a foundation modelvia a commercial API).
Another paradigm adds small trainable parameters to the original model,either within its layers or on top of its embeddings.
These include linear probes (linear classifiers)
[54]and adapters (small bottleneck MLPs) [29, 52, 53, 55].
Recently, Kumar et al.
[38], Wortsman et al.[68] propose methods that use linear probes to improve robustness to dataset-level out-of-distributionshifts.
Gao et al.
[21] propose to train single adapters on top of FM embeddings to improve averagedownstream task accuracy.
We focus on group shifts that occur within a dataset.
We also show that thelatter can hurt group robustness, and propose alternatives to consistently improve group robustness.
Problem
In Section 3.1, we first describe the group robustness problem setting.
In Section 3.2, we illustrate thisproblem with foundation models.
We show that zero-shot classification with foundation models, andexisting baseline approaches to improve downstream inference, can result in poor group robustness.
3.1 Preliminaries: group robustness and task setup
We emphasize robustness to distribution shifts between groups in this work.
For setup, we followprior works [36, 42, 58, 64] that alternatively describe the phenomenon as hidden stratification [64] orsubpopulation shift [36].
For some task, we have N samples { ( x i , y i , g i ) } N i =1 , with sample featuresor inputs x i ∈ X , class labels y i ∈ Y , and group labels g
i ∈
G .
Let C = |Y| be the number ofclasses.
We use g i to indicate the group that each sample belongs in, but do not observe group labelsduring training.
Distribution shifts may occur between samples in different groups but the same class.
Every sample ( x i , y i , g i ) is drawn from some unknown joint distribution P .
Let P g be the specificdistribution conditioned on g for any g ∈ G .
For classification loss ℓ : Y × Y (cid:55)→ R and classifierf θ : X (cid:55)→ Y , we want f θ to be accurate, i.e . achieving low average error:
[Equation]
and group robust , i.e ., achieving a small gap between its average error and its worst-group error:
[Equation]
Different from domain generalization or out-of-distribution (OOD) evaluation, we observe each groupin training, validation, and test splits.
However, standard training via empirical risk minimization(ERM) can still lead to poor test set group robustness because training groups may be imbalanced[58, 64, 71].
Here, foundation models are not trained on the training data, but we show that zero-shotclassification with foundation models can still result in poor group robustness.
findings
To motivate the rest of this work, we now demonstrate the group robustness problem with foundationmodels.
We first describe different natural group shifts for evaluation.
We next detail primarybaseline approaches.
We finally summarize our findings after evaluating these baselines on 11popular foundation models across 9 standard group robustness benchmarks used in prior work[36, 44, 56, 58, 60].
We present four representative scenarios based on training data assumptions andgroup robustness outcome.
Critically, we find that zero-shot classification with foundation modelsmay result in poor group robustness.
We also find that baseline methods to improve downstreamtransfer do not consistently improve group robustness, and can make group robustness worse.
Dataset group shifts.
We benchmark methods on the following sources of group shift (Figure 2):
• Spurious confounders.
We evaluate across groups which may or may not carry spurious
confounders—input features predictive for some, but not all groups in a class.
For example,
in Waterbirds [58, 67], a water background is a confounder for the waterbirds class.
• Subclass variance.
We evaluate across groups which are different fine-grained subclasses.
For
example, in BREEDS Living-17 [60], the ape class includes images of gibbons and gorillas.
• Data source variance.
We evaluate across groups which are the same class but sourced from
different datasets.
For example, we set up the CIFAR-10.02 dataset by combining CIFAR-10 [37]and CIFAR-10.2 [44].
The airplanes class contains samples from both datasets.
Baseline methods.
To evaluate foundation model group robustness, we consider the
following
baseline methods.
Following prior work [19, 32, 41, 45, 54], for all approaches we first compute
N sample embeddings and C class embeddings using a foundation model.
With foundation modelembedding dimension D , let u n ∈ R D be a sample embedding and c n ∈ R D be a class embedding.
• Zero-shot classification [54]: We classify each sample via the nearest class embedding to its
sample embedding u n .
Specifically, we compute the class-wise logits for each sample x n as
[Equation]
where ˆ u n = u n / ∥ u n ∥ is the ( ℓ - 2 ) normalized sample embedding of x n , ˆ W ∈ R D × C is a matrixwhose columns are the normalized class embeddings { ˆ v c } Cc =1 , and τ is a temperature parameter.
The highest class logit corresponds to the nearest neighbor and largest dot product.
As standard,for class embeddings we convert each class name to a natural language prompt, e.g ., “photo of a[class name] ”, and feed the tokenized prompt to a foundation model’s text encoder.
As in priorwork [54], we engineer class prompts by trying several templates.
We defer details to Appendix A.2,such as optimal templates (Table 10) and a list of all templates tried (Table 20).
• Linear Probe [54, 68]: We train a linear classifier on top of training data sample embeddings.
Specifically, with classifier f θ ( u ) =
W ⊤ u , we update the weights W ∈ R D × C with a cross-entropyloss applied over training data sample embeddings { u n } Nn =1 and labels { y n } Nn =1 .
• Adapter [21, 55]: We train a single 2-layer bottleneck multilayer perception (MLP) to outputtransformed sample embeddings, which we use instead of the original sample embeddings to
classify with in the zero-shot procedure above.
Specifically, with adapter hidden-layer dimension H ,ReLU activation function σ , and adapter weights ϕ =
[ W 1 , W 2 ] —where W 1 ∈ R D
× H is a lineardown-projection and W 2 ∈
R
H
× D a linear up-projection—we compute “adapted” embeddings
[Equation]
We classify samples with the zero-shot class matrix ˆ W , temperature τ , and normalized adaptedembeddings ˆ f ϕ ( u ) =
f θ ( u ) / ∥ f ϕ ( u ) ∥ .
The final outputs are given by f θ ( u ; ˆ W, τ ) =
ˆ W ⊤ ˆ f ϕ ( u ) /τ .
Like with linear probes, we update ϕ with a cross-entropy loss using training data labels { y n } Nn =1and a softmax over the dot product-computed logits as class-wise probabilities.
[Table] [Table]
For evaluation
[Table]
we train both linear probes and adapters with standard empirical risk minimization(ERM)
[Table]
which aims to minimize the empirical risk: ˆ L ( f θ )
= 1 N (cid:80)
Nn
=1 ℓ ( f θ ( u n ) , y n ) .
Discussion and representative outcomes.
In Table 1, we report worst-group and average accuraciesalong with their corresponding gaps on four representative group robustness datasets, using zero-shotclassification, linear probes, and adapters on CLIP ResNet-50 embeddings.
We select datasets toreport based on training data setup and group robustness outcome, where we find that (i) the relativegroup size ratios, (ii) the type of group shift, and (iii) the choice of adapter or linear probe influencesgroup robustness improvements.
We note descriptive characteristics and outcomes in Table 2, andsummarize three main takeaways below.
Appendix A contains results for all datasets and models.
1 Foundation model zero-shot classification may not be group robust :
Across datasets, we find
that zero-shot classification with CLIP ResNet-50 embeddings can achieve 7.9 to 80.7 pp gaps
between worst-group and average accuracy.
Worryingly, poor group robustness is accompanied by
high average error (from 69.9% to 92.9%), the usual metric for evaluating zero-shot classification.
This further supports the importance of improving group robustness.
2 Efficient baselines do not consistently improve robustness : We find that while previously
proposed linear probes and adapters are efficient ways to improve accuracy on downstream tasks,
these benefits do not consistently carry over to improving group robustness.
• When training data is balanced, both linear probes and adapters can substantially improve
group robustness and worst-group accuracy (reducing the robustness gap by 43.2 and 54.
pp respectively on BREEDS Living-17).
However, when minority groups are rare, in someinstances, approaches can hurt group robustness.
On CelebA, adapters and linear probes
increase the gap by 50.2 and 74.9 pp, and reduce worst-group accuracy by 37.9 and 62.1 pp.
3 We can improve group robustness with only foundation model embeddings : Our positive
results suggest that poor zero-shot classification may not be because sample embeddings lack the
information required to classify groups correctly.
Rather, we may just require the right training
strategies to learn how to better classify by this information.
Altogether,
takeaways 1 and 2 motivate the need for methods to effectively improve robustness in thefoundation model setting.
Takeaway 3 suggests we can make progress on this problem.
Method
Having established the group robustness problem in Section 3, we now propose a simple contrastiveadapter training strategy to improve group robustness.
In Section 4.1, we setup our approach byidentifying possible sources of limitation with standard adapter training.
In Section 4.2, we then usethese insights to propose a simple yet effective approach that counteracts these limitations.
4.1 Understanding prior limitations via embedding metrics
To guide a first-step strategy for improving robustness, we first outline high-level reasoning for whyzero-shot and ERM-trained adapters fail to classify groups correctly.
Recall that a key propertyof group robust classification is that all sample embeddings belonging to the same class shouldembed closer to their ground-truth class embedding than any other class embedding.
If zero-shotclassification for a specific class is accurate on average but not group robust, then in the pretrainedfoundation model embedding space there exists groups that embed “close” to their ground-truth classembedding, and groups in the same class that embed “far away” (measured via cosine similarity).
One way to interpret standard adapter training with FM embeddings via ERM is that it aims to bringthese initially far apart sample embeddings closer to their ground-truth class embedding.
Restatingthe standard sample cross-entropy loss with adapters makes this clear as an InfoNCE loss [13, 51]:
[Equation]
[Equation]
with sample embedding u as an anchor, class embedding v of ground-truth y as a single positive, andthe other C − 1 class embeddings as negatives.
Via ERM of the sample cross-entropy loss, adaptersthus bring zero-shot-incorrect anchors closer to their class embedding positives (minimizing Eq. 5).
However, in Section 3 we found this loss works in some scenarios but not others.
Intuitively, Eq.
5can fail to bring samples closer to their correct class embedding ( e.g ., on CelebA).
To find additionalways to bring points together, we hypothesize that poor robustness also accompanies poor similaritybetween sample embeddings from different groups but the same class.
We verify this in Figure 3 byempirically measuring the average pairwise cosine similarity and group alignment loss L align [71]—which measures the pairwise Euclidean distance—between sample embeddings in the same class butdifferent groups.
We compare these metrics with embeddings computed with trained adapters andthe initial foundation model embeddings, and find that higher worst-group accuracy corresponds tohigher cosine similarity and lower alignment loss between groups in the same class.
4.2 Approach: Contrastive Adapting
To improve robustness, we therefore propose to more effectively bring far away samples togetherby introducing greater training signal via other sample embeddings.
Instead of limiting ourselves toa single class embedding positive and a limited set of C − 1 negatives, we expand our positives byincluding sample embeddings for points in the same class far away from the anchors among pretrainedembeddings ( e.g ., likely in different groups).
We expand our negatives with sample embeddingsfrom different classes.
Following prior work [22, 71] that finds sampling hard negatives beneficialfor robust contrastive learning, we also use the computed foundation model sample embeddings tosample negatives from points nearest to the anchors but in different classes.
As the number of trainingdata points N is often much larger than the number of classes C , these choices are further supportedby prior work suggesting more positives and negatives are beneficial for contrastive learning [34, 57].
In practice, contrastive adapting is simple to implement with three components:
• Foundation model embedding and prediction : We compute FM embeddings over labeled trainingdata.
To guide sampling, we collect zero-shot predictions over this data.
• Contrastive sampling :
For each class, we identify an “anchor” sample embedding u ∈ U thatzero-shot predicts incorrectly, and P “positive” sample embeddings P ( u ) that zero-shot classifiescorrectly.
We do this as a heuristic for finding samples “far apart” in the FM embedding space, sopushing them together improves robustness over zero-shot.
We also identify M hard “negative”sample embeddings M ( u ) by computing the nearest neighbors to the anchors in different classes,
using cosine similarity between the sample embeddings.
• Training objective : We use a supervised contrastive loss [34] with the sample embeddings, i.e .
[Equation]
We also use a standard cross-entropy
loss
over minibatches of sample embeddings and their classembeddings.
This aims to keep
adapted embeddings close
to
their ground
-truth
class embeddings.
To avoid undoing Eq. 6 and push “far away” points closer to their ground-truth class embeddings,we upsample the number of zero-shot incorrect samples to equal the number of zero-shot correctsamples in each minibatch.
We thus use contrastive supervision from class and sample embeddings.
Robust generalization with adapters.
The contrastive loss in Equation 6 is also supported by recentresults suggesting that minimizing the class-wise alignment loss L align helps bound the worst-groupversus average error gap for that class ( cf .
Thm 3.1, Zhang et al.
The bound however scales withthe Lipschitz constant of the neural network, and upper bounds for estimating this constant can growwith the size of the network [18, 66].
However, as our adapters are small 2-layer MLPs, estimates ofthis constant suggest we can obtain better generalization with fewer training samples [24, 33, 48, 70].
In Section 5.2, we later show this corresponds to better data efficiency.
Experiments
We now validate that contrastive adapting enables effective and efficient group robustness.
First, in
Section 5.1, we evaluate the effectiveness of contrastive adapting against efficient methods to improve
FM inference.
We study whether the approach consistently improves worst-group accuracy andgroup robustness over zero-shot classification, how contrastive adapting compares against additionalefficient methods that only require foundation model embeddings, and whether contrastive adaptingscales to a variety of foundation model architectures.
Next, in Section 5.2, we study the efficiency ofcontrastive adapting against effective group robustness approaches.
We find that the prior robustnessgains are not only relative to other efficient FM training methods; contrastive adapting also enablesstate-of-the-art robustness on popular benchmarks, but with greater parameter and data efficiency.
5.1 Robustness comparison for efficient foundation model methods
To first judge the effectiveness of contrastive adapting, we evaluate the method across the same set ofinitial robustness benchmarks and foundation model architectures discussed in Section 3.
As in priorgroup robustness evaluation, we do not assume training groups labels, but do assume group labels invalidation data for hyperparameter tuning and model selection [36].
We include experimental detailsfor all models and hyperparameters in Appendix C.
As baselines, we compare against zero-shot classification [54], ERM linear probing [38, 54], and
ERM adapter training [21].
We also compare against recent methods designed to improve downstreamtransfer in related settings, while similarly only requiring pretrained model embeddings:
• Weight space ensembling (WiSE-FT) [68], which first trains a linear classifier with standard ERM,and then ensembles the classifier outputs with the initial zero-shot predictions.
While proposed forboth training linear classifiers and finetuning the original weights of a foundation model, we focuson the linear classifier version for fair comparison in our setting.•
Deep feature reweighting (DFR) [35], which first trains a linear probe on embeddings computedfrom a pretrained model over group-balanced data.
As we do not assume training group labels, wefirst infer groups using zero-shot classification with foundation model embeddings.
As in priorwork [42, 71], we treat the incorrect and correctly classified samples as proxies for different groups.
[Table]
C L I P
V i T - L /
[Table] [Table]
Finally, if we have validation group labels, we plausibly know what groups are in the test data.
Wethus also compare against group-informed prompting (Group Prompt ZS), which performs zero-shotclassification using prompts with group information ( e.g ., “a waterbird on a land background”).
Consistent robustness improvements over zero-shot.
In Figure 4 we report contrastive adapting’srelative gains in worst-group accuracy over zero-shot classification on all 9 robustness benchmarks.
Unlike prior adapter training approaches, contrastive adapting consistently improves group robustnessover zero-shot classification, achieving 8.5 to 56.0 pp higher worst-group accuracy.
Representative dataset evaluation.
In Table 3 weevaluate how contrastive adapting compares againstadditional lightweight methods for improving robustness.
We evaluate with group-imbalanced and groupbalanced training data across spurious confounder,subclass, and data source group shifts, using embeddings from CLIP ResNet-50 (RN-50) and CLIP ViTL/14.
On average, contrastive adapters also improveworst-group accuracy by 12.4 and 4.1 pp over the nextbest methods on CLIP RN-50 and ViT-L/14 models.
Transfer across architectures.
To further judge contrastive adapting’s effectiveness, we evaluate howthe prior improvements transfers to different foundation models.
In Table 4, we find that contrastiveadapters substantially improve group robustness forother foundation models such as CLOOB [19].
Contrastive adapting also scales across foundation modelbackbones, raising worst-group accuracy by 33.7 to 61.7 pp while only training models with 0.52%to 1.03% of the original FM parameter count.
5.2 Measuring efficiency among effective group robustness solutions
In the previous section, we showed how contrastive adapters could significantly improve grouprobustness for foundation models.
We now expand on contrastive adapting’s efficiency.
We find
[Table]
that for group robust classification in general, contrastive adapting can achieve state-of-the-artperformance despite only training ≤ 1% of the usual model parameters.
The lightweight nature ofcontrastive adapting also leads to better data efficiency than existing state-of-the-art approaches.
Robustness comparison to state-of-the-art methods.
In Table 5, we evaluate how contrastiveadapting with CLIP RN-50 and ViT-L/14 embeddings compares to current state-of-the-art robustnesstechniques.
For evaluation, we use the popular Waterbirds and CelebA datasets.
The existing methodstrain an ImageNet-pretrained ResNet-50.
We find that on both datasets, contrastive adapting achievescomparable worst-group accuracy to recent group robustness methods, despite only training ≤ 1%of their parameters.
Notably, contrastive adapting outperforms some of these approaches by up to5.0 and 10.1 pp for Waterbirds and CelebA respectively.
Contrastive adapting also falls short of thestate-of-the-art Spread Spurious Attribute (SSA) method by 2.1 pp on Waterbirds, but achieves 0.2higher worst-group accuracy on CelebA. These results suggest that contrastive adapters are not onlyeffective at improving group robustness for foundation models, but also enable competitive robustclassification in general at a fraction of prior approaches’ trainable parameter counts.
Data efficiency evaluation.
Beyond model parameter count, we also study if the lightweight nature ofcontrastive adapting transfers to better data efficiency.
We compare contrastive adapting to the best performing SSA on subsampled versions of Waterbirds.
Toevaluate how well methods maintain group robustness with less training data available, we keep groupratios preserved ( i.e ., 1% of all training samples belongs to the smallest Waterbirds group [58]).
In Figure 5, we show that contrastive adapting substantially outperforms SSA in lower data regimes.
With only 100 and 250 training samples, contrastiveadapting outperforms SSA-trained ResNet-50s by42.6 and 25.9 pp.
With 100 training samples, contrastive adapting still achieves 20.8 pp higher worstgroup accuracy than zero-shot classification with only1 training sample in the smallest Waterbirds group.
In contrast, SSA’s accuracy drops significantly, resulting in 17.6 pp lower worst-group accuracythan zero-shot classification.
To connect this fewer data result to our generalization discussion in
Section 4.2, with prior methods [18] we estimate the trained adapter Lipschitz constant.
We estimatethe constant to be 29.3, much lower than that reported for larger networks ( e.g ., ResNet-50s) [18, 66].
Conclusion
349 We study the group robustness of popular foundation models.
We find their zero-shot classificationmay not be robust to various group shifts, establish that baseline linear probe and adapter strategies donot reliably improve robustness, and propose a simple adapter strategy to significantly and consistentlyimprove FM robustness without finetuning.
This suggests FM embeddings do contain group-relevantinformation, and we show that we can use FM embeddings to efficiently achieve state-of-the-art robustclassification.
We recognize the limitations of computational solutions to subgroup performancedisparities, and the need to understand FMs in broader socio-technical systems [8].
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Apppendix
• Did you include the license to the code and datasets?
[Yes] See Apppendix•
Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For
all
authors
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] See Appendix.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] See
Appendix.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] We do not
include theoretical results
(b) Did you include complete proofs of all theoretical results?
[N/A] Not Applicable
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] See attached
zip in supplementary.
(b) Did you specify all the training details (e.g., data splits, hyperparameters,
how they
were chosen)?
[Yes] See Appendix.
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] For space we defer these to the appendix.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] For space we defer to the appendix.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Appendix.
(b) Did you mention the license of the assets?
[Yes] See Appendix.
(c) Did you include any new assets either in the supplemental material or as a URL?
[N/A]
We only use existing assets, which we discuss in the appendix.
(d)
Did you
discuss whether
and how consent was obtained from people whose datayou’re
using
/curating?
[No] We use existing popular robustness benchmarks for allour datasets.
While some involve people (CelebA, CivilComments-WILDS, AmazonWILDS), we were not able to locate information on how this data was collectedregarding subject consent.
However, we do discuss the licenses and agreements set bythe original curators of these popular benchmarks.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
[Table]
information or offensive content?
[Yes] See Appendix.
[Table]
If you used crowdsourcing or conducted research with human subjects...
[Table]
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A]
We do not crowdsource or conduct research with human subjects.
[Table]
(b) Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[N/A]
We do not crowdsource or conductresearch with human subjects.
[Table]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
We do not crowdsource or conduct research
[Table]
with human subjects.
[Table]
A Expanded zero-shot evaluation for group robustness
In this section, we expand on the zero-shot evaluation of various foundation models on grouprobustness benchmarks discussed in Section 3.
We first describe the datasets and models used in
Appendix A.1.
We then include results in Appendix A.2.
We find consistent trends of poor grouprobustness with zero-shot classification, marked by poor worst-group accuracy and large gaps betweenaverage and worst-group accuracy.
A.1 Additional details on robustness datasets and foundation models
Datasets.
To benchmark zero-shot group robustness, we use a diverse set of datasets with groupshifts from prior robustness literature.
We describe them below and include details on size of groupsand type of group shift in Table 6:
• Waterbirds [58, 67].
We classify images by bird type.
Each class ∈ { waterbird , landbird }carries two groups: birds on water backgrounds, and birds on land backgrounds.•
CelebA [43, 58].
We classify images by celebrity hair color.
Each class ∈ { not blond , blond }carries two groups: celebrities labeled as male, and celebrities labeled as female.• BREEDS ( Living-17 , Nonliving-26 ) [60].
For the Living-17 and Nonliving-26 datasetsin the BREEDS benchmark sourced from ImageNet [60], we classify images by one of severalcategories.
Each class is a coarse category consisting of multiple fine-grained groups.
Groups inthe same class may be visually distinct ( e.g ., the ape class includes images of gibbons and gorillas).While the original benchmark evaluates how classifiers trained on seen source groups generalizeto unseen target groups
we adapt the datasets for our group robustness setting by adding 5% ofthe images in each target group to the source groups, and evaluating worst-group accuracy overall source and target groups.
• CIFAR-10.001 , CIFAR-10.02 [37, 44, 56].
We classify images by one of 10 categories.
Wecombine CIFAR-10 [37] and either CIFAR-10.1 [56] or CIFAR-10.2 [44], which are collected fromdifferent sources.
The new datasets’ classes carry two groups determined by the source dataset.
ForCIFAR-10.001, 2% of the combined train and validation data is sourced from CIFAR-10.1 (therest from CIFAR-10).
For CIFAR-10.02, 10% of the combined train and validation data is sourcedfrom CIFAR-10.
(the rest from CIFAR-10).
For both, we then split the combined data into 80%train and 20% validation sets.
We merely combine the official test splits.
[Table]
• FMoW-WILDS [14, 36].
We classify satellite images into one of 62 building or land-use categories( e.g ., airport , zoo ).
Each images belongs to one of five groups based on continental region.
Totest group robustness, we compare the accuracies over all samples in each group as in the WILDS
[Table]
benchmark [36].
We also evaluate only over test images from the same time period as trainingimages (the “IID” split in the original WILDS benchmark [36]).• CivilComments-WILDS [9, 36].
We classify if a text comment is toxic or not.
Samples areorganized into 8 groups based on mention of a demographic identity ( e.g ., “female”, “LGBTQ”).• Amazon-WILDS [36, 49].
We classify if an online text review is positive or negative.
Reviews areorganized into different groups based on the product category ( e.g ., books , electronics ).
Weadapt this dataset from the official Amazon-WILDS split by using the category_subpopulationsplit.
We also map the original class labels, which are star-ratings from 1 to 5, to positive ornegative reviews by discarding samples with a 3-star rating, and re-labeling 1- and 2-star ratings asnegative and
4- and 5-start ratings as positive .
[Table]
Foundation models.
For image datasets, we evaluate pretrained CLIP [54] and CLOOB [19]vision-language models using publicly available weights 23 .
We evaluate 7 available CLIP models:
ResNet image encoder backbones ( RN-50 , RN-101 , RN-50x4 ), and 5 Vision Transformer imageencoder backbones: ( ViT-B/32 , ViT-B/16 , ViT-L/14 , ViT-L/14@336px ) and 2 CLOOB models(all available: RN-50 , RN-50x4 ).
For text datasets, we evaluate 2 pretrained GPT-Neo [6] text modelstrained on the Pile [20] ( GPT-Neo-125M , GPT-Neo-1.3B ) available on HuggingFace 45 .
[Table]
https
[Table]
[Table] [Table]
at
[Table]
: https://huggingface.co/EleutherAI/gpt-neo-
[Table]
Neo 1.3B: https://huggingface.co/EleutherAI/gpt-neo-1.3B
A.2 Group robustness results
In Figure 6, we chart worst-group and average accuracies achieved by various zero-shot foundationmodels across the group robustness datasets.
Larger gaps between accuracies, i.e . high averageaccuracy yet low worst-group accuracy, indicate poor group robustness.
In aggregate, on all datasetsexcept FMoW-WILDS and Amazon-WILDS, we observe a shared pattern of noticeable gaps betweenaverage and worst-group accuracy, suggesting that zero-shot classification with popular foundationmodels may not be group robust.
We perform zero-shot classification as described in Section 3.
Asrecommended by Radford et al.
[54], for each dataset we consider several prompt templates.
Weengineer prompts by using the single best template based on validation worst-group accuracy.
In
Table 10, Appendix C.5 we include a list of optimal prompts used.
Table 20 includes a list of allprompt templates tried for each dataset.
[Table]
B Contrastive adapter implementation details
We provide further details on the adapter architecture and training sampling.
B.1 Adapter architecture
[Figure]
ReLU
[Figure]
Similar to prior works [21, 29], the adapters we use are bottleneck2-layer multilayer perceptrons (MLPs).
We set the input dimensionand output dimensions as the same as the pretrained foundationmodel embedding dimension, and pick a smaller dimension for thehidden layer (frequently 128, although this was chosen as a heuristicand not tuned).
We also experimented with using a single residualconnection [28] and batch normalization layer [31] between the inputand output layers, but only found the latter to be helpful.
Pytorch-likepseudocode is given below.
The adapter is visualized in Figure 7.
[Figure] [Figure]
[Figure] [Figure]
[Figure] [Figure]
B.2 Adapter training sampling
Recall in Section 4.2 that we train the adapter with both a supervised contrastive loss over specificallysampled “contrastive batches”, and a cross-entropy loss over resampled batches, both over the fixedpretrained foundation model embeddings.
We use the foundation model’s zero-shot classificationpredictions to guide sampling for both.
We outline the algorithms for sampling training batches in Algorithm 1 and Algorithm 2.
We thentrain an adapter by applying the contrastive loss (Eq. 6) and the cross-entropy loss (Eq. 5) over thesebatches in Algorithm 3.
M sample embeddings cosine similarity to u y m
=
y a randomly sampling M embeddings
C Additional experimental details
C.1 Model selection and hyperparameters
We describe the hyperparameters used for each dataset and method.
As in prior group robustnesswork [36], we select the best model and hyperparameters based on early stopping that achieveshighest worst-group validation accuracy.
For all methods and datasets, we train both linear probes andadapters with SGD, and sweep over learning rate ∈ { 1e-3, 1e-4, 1e-5 } and weight decay ∈ { 5e-5, 5e4, 1e-1 } .
For adapter classification, we used the default temperature used for zero-shot classificationin CLIP [54].
We did not tune the contrastive temperature.
Unless noted, we ran all numbers.
We list hyperparameters for linear probes (Table 7), adapters (Table 8, both ERM and contrastive),and contrastive-specific hyperparameters (Table 9).
We discuss method-specific hyperparameters:
• Contrastive adapting requires selecting three additional hyperparameters: the number of positivesand negatives, and the number of nearest neighbors to
sample negatives
from
For these
we
swept
over
the
following combinations of (number positives, number negatives, number neighbors):
(2048, 2048, 2146), (2048, 2048, 4096), (512, 512, 1024).• Weight-space ensembling (WiSE-FT): WiSE-FT requires picking a value α ∈ [0 , 1] to compute aweighted combination of the zero-shot classifier parameters and the trained linear probe parameters.
We sweep over intervals of size
[Table] [Table]
[Table] [Table]
C.2 Data splits
We use the same train, validation, and test splits for Waterbirds, CelebA, FMoW-WILDS, AmazonWILDS
[Table]
and CivilComments-WILDS as in prior work.
For BREEDS and CIFAR datasets that weadapt for our problem setting, we construct test splits by combining official test splits from theoriginal benchmarks.
We then create training and validation splits by combining the rest of the datafrom these benchmarks
[Table]
and randomly splitting this into 80% training data and 20% validation data.
No original test data is seen during training on our splits.
C.3 Additional dataset assets details and discussion
Dataset licenses.
To curate CIFAR-10.0001 we use the CIFAR-10.1 dataset, which is distributedunder the MIT License.
The FMoW-WILDS dataset is distributed under the FMoW Challenge Public
License 6 .
The CivilComments-WILDS dataset is distributed under CC0 1.0.
The Amazon-WILDSdataset does not have a license, but is requested to be used for research purposes only [36].
We werenot able to find explicit license information for CIFAR-10.2, Waterbirds, CelebA, or the BREEDSdatasets.
We note that the BREEDS datasets are sourced from ImageNet, which is distributed underthe BSD 3-Clause License, and set up with code from the MadryLab robustness GitHub repository 7 ,which is distributed under a MIT license.
The authors of the CelebA dataset provide a list ofagreements 8 , including that the dataset is used only for non-commercial research purposes.
Existing assets personally identifiable information and offensive content.
The CelebA datasetconsists of images of celebrity faces, which are personally identifiable.
The dataset is also categorized
[Table]
CLIP “This is a picture of a [class_name] .”
CLOOB “a [class_name] ”
[Table]
CLIP “A photo of a celebrity with blond hair.” CLOOB “A photo of a celebrity with blond hair.”
[Table]
CLIP “This is a picture of a [class_name] .” CLOOB “This is a picture of a [class_name] .”
[Table]
CLIP “A photo of a [class_name] .”
CLOOB “a [class_name] ”
[Table]
CLIP “a [class_name] ”
CLOOB “a [class_name] ”
[Table]
CLIP “a [class_name] ”
CLOOB “a [class_name] ”
[Table]
WILDS
CLIP “satellite view of the [class_name] ”
CLOOB “aerial view of an [class_name] ”
[Table]
WILDS
[Table]
“Negative”
CivilComments-WILDS
[Table]
“Not toxic” by male and female identification at the time of curation, which may be outdated.
The CivilCommentsWILDS contains text samples flagged as toxic by toxicity classifiers [36], which contain potentiallyoffensive content.
Both datasets are existing assets, and both personal identifiability for CelebA andoffensive content for CivilComments-WILDS can be checked by inspecting the original data inputs(images and text comments).
C.4 Compute and resources
[Table]
All experiments were run on a machine with 14 CPU cores and a single NVIDIA Tesla P100 GPU.
For training a contrastive adapter on top of CLIP ResNet-50 Waterbirds embeddings, this tookapproximately 30 minutes to run 100 epochs.
Other than the numbers reported from their originalpublications in Table 5, we report all numbers from running experiments on the same machine.
C.5 Class prompt templates
In Table 10, we list the templates used to generate class prompts for each dataset.
As a reminder,for each provided class name in a dataset, we create a prompt by inserting the class name into theprompt template.
We then encode this prompt with a foundation model text encoder to get classembeddings.
We selected these prompts for each dataset by trying several different prompt templatesper dataset, and selecting the best prompt template based on validation set worst-group accuracy.
Table 20 contains a full list of all templates tried.
D Additional related work discussion
We provide additional discussion of related work and connections to our work below.
Zero-shot classification with foundation models.
Our work builds on a growing literature onapplying foundation models, large pretrained models that can be applied to various downstream tasks.
These models demonstrate exciting promise in their ability to achieve accurate downstream transferwithout any additional finetuning [8, 10, 54].
In particular we consider the zero-shot capabilitiesof pretrained vision-language foundation models.
These models, such as CLIP [54], ALIGN [32],and CLOOB [19] are trained on massive amounts of naturally paired image-text data, e.g ., Internet images and their corresponding captions.
Consisting of an image encoder (usually a ResNet or Vision
Transformer) and a text encoder (usually a Transformer), such foundation models are commonlytrained to learn a shared image-text embedding space where embeddings of images are most similarto embeddings of their corresponding caption text.
While these objectives have been shown tolead to powerful representations [17, 73], a crucial element for successful zero-shot classificationis training data scale [54].
However, added scale can also be a double-edged sword; when zeroshot classification still makes undesirable mistakes, standard ways to correct for these mistakes viaretraining can become prohibitively expensive.
We study one such motivating instance via grouprobustness, and provide a first-step solution towards improving group robustness efficiently.
Robustness of foundation models.
Prior works have studied the robustness of foundation modelinference to natural distribution shifts.
Radford et al.
[54] show that zero-shot CLIP models canbe more robust to out-of-distribution (OOD) shifts than prior ImageNet-trained models, measuredvia better generalization to various dataset-level distribution shifts on ImageNet classes.
However,they also show that finetuning, or updating the original weights, of CLIP models on ImageNet canreduce this OOD robustness.
Kumar et al.
[38], Wortsman et al.
[68] thus propose finetuning methodsthat improve downstream in-distribution accuracy while maintaining out-of-distribution robustness.
Kumar et al.
[38] specifically study the trade-off between linear probing and finetuning, finding thatfinetuning on downstream data can improve generalization on in-distribution data over linear probingbut more substantially hurt performance OOD data than linear probing.
They show theoreticallyand empirically that a two-step strategy of first linear probing then full fine-tuning can combinethe performance boosts of both.
Wortsman et al.
[68] focus on the OOD trade-off presented by
Radford et al.
[54] between a finetuned foundation model and its pretrained zero-shot weights.
Theypropose weight-space ensembling (WiSE-FT), which computes a weighted average of the finetunedand pretraiend foundation model parameters, and show that the resulting averaged parameters can insome instances achieve higher performance on both data distributions that the model was finetunedon and unseen OOD data than the initial finetuned and zero-shot or pretrained models.
They showthis effect with both full finetuning and training a linear probe.
Unlike these works, we focus onfoundation model robustness to group shifts that occur within a dataset.
We also compare against thelinear probe version of WiSE-FT, and find that training adapters can be advantageous for achievinghigher group robustness on various datasets.
Recently, other works also study foundation model learned spurious correlations and biases.
Singlaet al.
[63] show how various models (including CLIP models) may rely on spurious artifacts to classify
ImageNet images.
Berg et al.
[5] aim to debias CLIP image embeddings of human faces using extrametadata (textual concepts or attributes) that the embeddings should ignore.
Our evaluation iscomplementary, noting poor group robustness across multiple types of data sources (objects, animals,human faces, text).
We also provide a method that works without additional training metadata.
Improving group robustness of deep learning models.
Improving the group robustness of deeplearning models is a common deep learning challenge, where models may learn biases during trainingthat lead to poor performance on certain groups.
This is a widespread issue presented in contextsranging from algorithmic fairness to healthcare diagnosis [7, 11, 26].
Several methods exist toimprove group robustness.
We compare against several recent approaches in Section 5.2.
Whileone effective strategy to improve group robustness is to upweight the error of worst-performinggroup during training [58, 59], training group labels may be impractical to obtain in practice [50, 64].
We thus consider robustness approaches which aim to work without training group labels.
Severalapproaches involve training two models; one model is first trained with standard ERM to help infergroups, and another trained with a robust objective using these inferred groups.
Just Train Twice(JTT)
[42] treats samples that the first model misclassifies as inferred minority group samples toupweight.
JTT then upweights these samples by a hyperparameter factor, and trains a second modelwith ERM on this upsampled data.
Environment Inference for Invariant Learning (EIIL) [15] infersgroups by assigning samples to group under which the ERM model maximally violates an Invariant
Risk Minimization [2] principle.
It then trains a robust model with Group DRO [58] using theinferred groups, which dynamically upweights the worst-performing groups during training.
CorrectN-Contrast (CNC) [71] instead identifies samples with the same class labels but different ERM modelpredictions, and trains a robust model by using a contrastive loss to learn similar representationsbetween these samples.
Spread Spurious Attribute (SSA) [47] specifically trains the first model topredict groups using a small set of group labels, before using Group DRO to train a robust model.
Contrastive Input Morphing (CIM) [65] trains a network to transform the input features of an image to better present class-specific information shared across groups.
Idrissi et al.
[30] suggest that simplychanging the training data by subsampling large classes (SUBY) or balancing the class samplingprobabilities (RWY), then training a model with ERM, can also improve group robustness.
E Additional experimental results
E.1 Extended main results
In Table 11 and Table 12 we report group robustness results evaluating all methods discussed in
Section 5.1 on all group robustness benchmarks.
Table 11 contains results for image datasets,using CLIP-RN50 embeddings.
Table 12 contains results for text datasets, using GPT-Neo 1.3Bembeddings.
As in Table 3, we report the worst-group and average accuracies, along with their gap.
Higher worst-group accuracy and smaller accuracy gap are indicative of better group robustness.
Allresults are computed over three random seeds, with mean and one standard deviation included (errorbars deferred to here from the main paper).
Compared to alternative methods, contrastive adaptingconsistently improves group robustness over zero-shot classification, and obtains highest worst-groupaccuracy and smallest accuracy gap on datasets where training adapters with ERM fails.
On datasetswhere ERM-trained adapters achieve best group robustness, contrastive adapters are also competitiveor closest to ERM-trained adapters among other robustness methods.
E.2 Contrastive adapter ablations
In this section, we ablate different training components of contrastive adapting.
We first study howablating the training objective (cross-entropy and contrastive losses) affects worst-group accuracyacross pretrained model embeddings on the Waterbirds dataset.
We then compare how differentcombinations of the full objective and hard sampling strategy impact robustness across each benchmark in Table 3.
Finally, we study how the number of positives and negatives used in contrastivesampling affects performance.
Our results suggest that the full combination of contrastive objectiveand hard sampling corresponds to best group robustness, and that contrastive adapters also benefitfrom training with larger batches of positive and negative samples.
E.2.1 Ablation on loss components in training objective
We first study the importance of the contrastive and cross-entropy components in contrastive adapting.
For evaluation, we use the Waterbirds dataset, and run ablations comparing adapters trained on top of
CLIP embeddings with (i) no contrastive component (Eq. 6), (ii) no cross-entropy component (Eq.
5),or the default proposed approach.
We evaluate across five different CLIP models and three seeds.
Wekeep all other training procedures consistent ( e.g ., we use the proposed “hard” sampling strategy).
In Table 13, we report worst-group accuracies.
We find that both contrastive and cross-entropycomponents are necessary for best worst-group accuracy.
The contrastive objective leads to asubstantial improvement over just the resampled cross-entropy loss (+17.9 pp on average).
However,we also note that without the cross-entropy objective to learn sample embeddings close to theirground-truth class embeddings, we observe high variance in classification accuracy.
We improve+26.9 pp on average using both objectives compared to contrastive alone.
E.2.2 Ablation on sampling strategy and contrastive training objective
We next explore how our proposed “hard” sampling strategy for training adapters affects performancein relation to using the contrastive objective.
We aim to understand if either component is morebeneficial, i.e ., can we just train adapters with the contrastive loss but random sampling (similar to
SupCon [34])?
Conversely, can we remove the contrastive loss just use the cross-entropy loss (Eq. 5)over
the “hard” resampled data?
We evaluate training adapters with CLIP RN-50 embeddings overthe four representative group-shift benchmarks in our main results in Section 5.1.
In Table 14, we report worst-group and average accuracies after individually ablating the hardsampling strategy and contrastive loss component.
On all datasets except CelebA, just trainingadapters contrastively leads to higher worst-group accuracy and group robustness than just doing thehard sampling procedure.
However, on Waterbirds and CelebA, where standard adapter training ( i.e .,
ERM Adapters), is a relatively poor baseline (c.f. Table 3) having both components substantially
[Table] [Table]
improves robustness.
Meanwhile, interestingly on BREEDS Living-17 and CIFAR-10.02, where
ERM adapters perform best across all methods (Table 3), taking away hard sampling also improvescontrastive adapter performance.
On these datasets, the random sampling in ERM may be beneficialfor learning.
Contrastive adapters can thus also benefit from random sampling in these settings
[Table] [Table]
[Table] [Table]
[Table] [Table]
E.2.3 Effect of contrastive batch size
While one advantage of training adapters is that because we train on embeddings, the memory sizeof our data inputs during training is much smaller than the traditional alternative ( e.g ., storing antensorized image).
We can thus train with larger batch sizes.
Here we study how contrastive batchsize, i.e .
how many positives and negatives we sample per anchor, affects worst-group accuracy.
Onthe Waterbirds and CelebA datasets and with CLIP RN-50 embeddings, we train a contrastive adapterwith varying levels of positives and negatives.
For both datasets, the default is 2048 positives and 2048negatives per batch.
We ablate these numbers with the following (positive, negative) combinations:(1, 1), (2, 2), (256, 256), (256, 512), (512, 256), (512, 512), (512, 1024), (1024, 512), (1024, 1024),(1024, 2048), (2048, 1024).
In Figure 8, we plot the effect of smaller batch sizes on worst-group accuracy.
We find that larger batchsizes weakly correspond to higher worst-group accuracy on both Waterbirds and CelebA.
However,perhaps surprisingly, we still maintain a substantial improvement over zero-shot classification withjust a single positive and negative per anchor.
E.3 Comparison to TIP-adapter and training sample nearest-neighbors lookup
On representative benchmarks, we perform further comparison to the nearest-neighbor look-upapproach employed by TIP Adapter [72].
Instead of learning transformed representations of pretrainedembeddings, another approach to better classify a test sample is to use the class of its nearest trainingsample.
Under the assumption that the training and test data are sampled from the same broaderdistribution and share the same groups, then test samples in a given group should embed closest totraining samples in the same group.
The training sample ground-truth class should then apply tothe test sample.
TIP adapter operates accordingly, keeping a cache of training sample embeddingsavailable at test-time.
One advantage is this allows for potentially more accurate classification withoutany training.
To test how well this idea fares for group robust classification, for each test sample weperform a look-up with all training samples, using cosine similarity to identify nearest neighbors.
In Table 15, we compare TIP-adapter with zero-shot classification and contrastive adapting on the
Waterbirds, CelebA, BREEDS Living-17, and CIFAR-10.02 group robustness benchmarks.
Forall methods, we use CLIP RN-50 pretrained embeddings.
We find that TIP adapter improvesworst-group accuracy over zero-shot classification on 3 out of 4 datasets, and notably achieves bestworst-group accuracy on BREEDS Living-17 without training any additional parameters.
However,the improvements are more marginal on Waterbirds and CIFAR-10.02.
Contrastive adapting still
[Table]
achieves 30.4 pp higher worst-group accuracy over TIP adapter on average.
This may suggest thatlearning a nonlinear transformation of the pretrained embeddings can still be helpful for better“presenting” class-specific information to classify samples by.
E.4 Evaluation with respect to weight-space ensembling trade-off
To provide additional perspective on how different embedding-only methods trade-off worst-groupand average accuracy, we compare how these methods perform with respect to the accuracy tradeoff traced out by weight-space ensembles.
Wortsman et al.
[68] show an interesting phenomenonwhere simply taking a weighted average of a trained linear probe and the original foundation
model(either over the weights, or the outputs) can result in a “pareto frontier” of accuracy metrics.
Theyspecifically show that a weight-space ensemble can often achieve better OOD performance withoutsacrificing too much IID performance.
compared to a single linear probe.
In this context, we seehow this trade-off occurs over average accuracy and worst-group accuracy across our representativeset of group-robustness datasets.
We also evaluate how other approaches (ERM adapters, DFR [35],contrastive adapters) fare along this trade-off.
In Figure 9, we plot the accuracies of these methods run on CLIP RN-50 embeddings.
We noteseveral observations.
Weight-space ensembles (WiSE-FT) achieve the desired effect on certaindatasets but not others.
On CelebA and CIFAR-10.02, we find that an ensemble can obtain a betterworst-group accuracy versus average accuracy trade-off than either zero-shot classification or linearprobes.
However, the single linear probe does at least as well as any ensemble in BREEDS Living-17,while the zero-shot classification does at least as well as any ensemble in Waterbirds.
We also find that among other methods, contrastive adapting is the only evaluated approach thatconsistently achieves higher worst-group accuracy than any weight-space ensemble.
While contrastiveadapting places “above” the trade-off curve traced out by WiSE-FT on 3 out of 4 datasets (CelebA,
BREEDS Living-17, and CIFAR-10.02), it tends to degrade average performance in favor of higherworst-group performance compared to other approaches.
Further work can improve on how to raiseworst-group performance without sacrificing any average performance when compared to zero-shotclassification or ERM-trained adapters.
E.5 Comparison to recent robustness methods adapted for the pretrained embedding setting
The scope of this paper focused on identifying the poor group robustness of foundation modelsand determining how to efficiently improve this robustness.
Thus, to judge the effectiveness ofour proposed approach, our main evaluation considered existing methods for efficiently improvingpretrained model inference, i.e ., with no model retraining and only access to pretrained embeddings.
Here, in an initial study on how we can adapt more robustness techniques for efficiently improvinggroup robustness, we consider how recent robustness methods proposed for standard model training(as introduced in Section 5.2) can transfer to the pretrained model setting.
We adapt methods suchas Just Train Twice (JTT) [42], Correct-N-Contrast (CNC) [71], and Just Mix Once (JM1) [23] totrain adapters, and compare these methods to the proposed contrastive adapting on the representativebenchmarks in the main results (Section 5.1).
In Section E.5.1, we describe how we adapt the methodto the pretrained model setting.
In Section E.5.2, we discuss key hyperparameters and our sweeps.
Finally, in Section E.5.3 we report results and comparison takeaways.
E.5.1 Method overview for adapters
• Just Train Twice (JTT) [42]: Following the two-stage process in JTT, we first train an initial ERMadapter (via the cross-entropy loss in Eq. 5) for a few epochs and upsample the samples
that the
adapter gets incorrect.
We then train a new adapter with the same objective but on the resampled
data.
Alternatively, we can also use the zero-shot pretrained model predictions for sampling.
• Correct-N-Contrast (CNC) [71]: Like JTT, CNC also employs a two-stage process.
We first
train an initial ERM adapter like in JTT, but instead of upsampling the incorrect samples, use
the first ERM adapter’s
predictions to set up a contrastive learning dataset in the second stage.
Following [71], we obtain contrastive batches by sampling anchors and positives as samples withthe same class, but different initial ERM adapter predictions.
We identify anchors and negatives
as samples with different classes, but the same initial ERM adapter predictions.
We then train a
second adapter with the contrastive loss in Eq.
6 on this training set.
Alternatively, we can also use
the zero-shot pretrained model predictions for sampling.
• Just Mix Once (JM1) [23]: We similarly first identify samples with the same class but different
predictions, using predictions from either training an initial ERM adapter for a few epochs, or
using the pretrained model zero-shot predictions.
Then to train a robust adapter, during each
training epoch we compute a set of interpolated sample embeddings by interpolating embeddingsfor samples in the same class but different predictions ( i.e . u
= αu 1 + (1 − α )
u 2 if u 1 , u 2 arepretrained sample embeddings that fit the criteria).
We follow the proposed strategy [23] forrandomly sampling α , alternating between a Uniform(0, 1) and Beta(2, 5) distribution.
As theembeddings have the same class, we use the original class label as the interpolated sample label.
We train another adapter on the interpolated embeddings using the cross-entropy loss (Eq. 5).
E.5.2 Training details and hyperparameters
We evaluate the group robustness after training adapters on CLIP ResNet-50 pretrained embeddings
with each method.
For all methods, we sweep over the same adapter hyperparameter set in Section C.1.
As each method trains an initial “stage 1” ERM model, before
using this model’s predictions tosetup a robust training stage, we also try this.
In addition to using the pretrained model’s zero-shot
[Table] [Table]
predictions, as a hyperparameter we try sourcing predictions from an initial ERM adapter aftertraining for 1, 2, 5, or 10 epochs.
In Table 16, we list the optimal hyperparameters for each methodand dataset.
We use the same learning rate, weight decay, momemtum, and optimizer for bothadapters.
We select models with early stopping based on best worst-group validation accuracy.
Whenthe zero-shot predictions lead to best validation set accuracy, we denote this by Stage 1 Epochs = 0.
Following recommendations for fair group robustness comparison [25], we try to keep the total sizeof all hyperparameter combinations consistent across methods.
Thus for JTT, we do not tune theupsampling ratio; we upsample samples by the class-specific ratios that equate the number of correctand incorrect samples per class (intuitively balancing groups to improve group robustness).
For CNC,we pick the same number of positives and negatives for each dataset as in Table 9.
E.5.3 Results
In Table 17 we report results on representative group robustness benchmarks, comparing contrastiveadapting to recent robustness techniques applied to training adapters.
We find that among methods,the proposed contrastive adapting strategy obtains best or competitive group robustness on all datasets.
For three out of four datasets (Waterbirds, CelebA, and BREEDS Living-17), contrastive adaptingobtains best worst-group accuracy, and on average across all datasets outperforms the next bestmethod by 1.7 pp.
Contrastive adapting also obtains competitive gaps between worst-group andaverage accuracy.
Notably, while the application of CNC to adapters also involves supervisedcontrastive learning over sample embeddings, contrastive adapting outperforms CNC on all datasets.
We preliminarily hypothesize that the negative sampling procedure in contrastive adapting, whichfocuses on the nearest neighbors of each anchor, and not just incorrectly predicted samples with thesame class (as in CNC), samples “harder” negatives that the initial pretrained model embeddingsmore erroneously embeds closer to the anchors.
By focusing on “pulling” these sample embeddingsapart, contrastive adapting may thus more effectively transform the pretrained representations forbetter group robust classification, as discussed in Section 4.1.
Building on these results and our initial proposed method, we believe (1) a more comprehensive evaluation of how well existing robust training methods can transfer to efficiently improving foundationmodel inference, and (2) a deeper exploration into how method differences impact performance fortraining on large pretrained model embeddings, are interesting grounds for future work.
E.6 Additional study on
how dataset properties impact adapter robustness
In this section, we try to further understand what properties of the representative benchmarks inthe main results (Section 5.1) may explain why ERM adapters achieve better group robustness on
BREEDS Living-17 and CIFAR-10.02 than contrastive adapters (c.f. Table 3).
Our study also helpsunderstand why contrastive adapters work significantly better on Waterbirds and CelebA.
We consider two hypotheses for the relative performance improvement of ERM adapters on CIFAR10.02 and BREEDS Living-17, but not Waterbirds and CelebA. First, in Appendix E.6.1, we discusshow the lack of group shift in CIFAR-10.02 may result in ERM adapters and linear probes performingbest among all methods.
Second in Appendix E.6.2, we explore how the group balance in the
BREEDS Living-17 training data may also enable ERM adapters to best improve group robustness.
In each subsection, we provide empirical validation supporting or opposing these hypotheses.
Ourfindings suggest that rather than these settings being challenging for contrastive adapting, the CIFAR10.02 and BREEDS Living-17 datasets as-is are more optimal for ERM training to achieve goodgroup-robustness.
When we make the BREEDS Living-17 task harder for group-robust classification,we find that contrastive adapting outperforms ERM training.
Furthermore, this harder BREEDS taskexhibits the same combination of significant group shift and group imbalance present in Waterbirdsand CelebA (where contrastive adapting outperforms ERM training by a large margin), suggestingthat contrastive adapters work particularly well in these harder group robustness settings.
E.6.1 Lack of group shift in CIFAR-10.
First, for CIFAR-10.02, we hypothesize that ERM adapters achieve high worst-group and averageaccuracy on CIFAR-10.02 because there are not significant distribution shifts between individualgroups in each class.
While we curated benchmarks to intentionally include different groups andinduced minority groups in CIFAR-10.02, if there are not significant group shifts between differentgroups, then training via ERM on all of the data may still allow adapters to learn generalizablecorrelations from the majority groups that apply to all groups. Thus using all of the available data asin ERM can be optimal.
Meanwhile, the contrastive adapter protocol may restrict training to a certainsubset of the available data via the hard sampling strategy, relatively hurting performance.
To test this hypothesis, we study how well adapters trained via ERM on datasets with only majoritygroups transfer to minority groups.
We subsample training sets such that 100% of all samples belongto majority groups (or the source group in BREEDS Living-17), such that models never see the othergroups during training.
We evaluate these models on the regular test sets, and report average andworst-group accuracy.
If models still obtain high worst-group accuracy, then this suggests that thereis little group shift between majority and minority groups.
The datasets are thus “easier” from agroup-robustness standpoint.
In Table 18, we compare results training ERM adapters on the subsampled datasets and the originalsplits (all trained on CLIP ResNet-50 pretrained embeddings).
Notably, on CIFAR-10.02, despitenever seeing certain groups during training, worst-group accuracy only drops 6.3 pp, and is stillsignificantly higher than chance (62.5% vs. 10.0% given 10 possible classes).
This suggests there islittle group-shift, as training on only the majority groups still enables sufficient transfer to minoritygroups.
Meanwhile, on Waterbirds, CelebA, and BREEDS Living-17, we see much larger drops inperformance.
This suggests these datasets exhibit more significant distribution shifts between groups.
E.6.2 Balanced groups in BREEDS Living-17
Next, for BREEDS Living-17, we hypothesize that the natural group balance in the training set allows
ERM adapters to achieve best or competitive worst-group and average accuracy.
Recall that thesame groups are encountered during train and test (for all splits we combine source and targetsplits from the original benchmark [60]).
This amounts to roughly evenly sized groups (c.f.
Table 6).
Thus if there are no actual minority groups in the training data, then training adapters with ERM tominimize the average empirical error across samples can also lead to good generalization across allgroups at test time.
We note that if the embeddings actually do contain the necessary information toclassify each group, then this can still happen despite the initial pretrained embeddings performingpoorly ( e.g ., by learning nonlinear transformations with the adapters).
Meanwhile, the samplingprocedure with contrastive adapting may restrict the training sample size or focus on certain groups,leading to poorer overall generalization.
[Table]
group-imbalanced setting.
[Table]
To test this hypothesis, we introduce group imbalance by subsampling certain groups in the trainingdata.
We downsample the training data such that there exist groups that only makes up 5% ofthe samples in their classes.
If group balance is a key factor for the robust performance of ERMadapters on BREEDS Living-17, then we expect the worst-group accuracy for ERM adapters to dropsubstantially.
By introducing group imbalance, but keeping the group shifts, we arguably also makegroup-robust classification more difficult on the BREEDS Living-17 dataset.
In Table 19, using CLIP ResNet-50 and CLIP ViT-L/14 pretrained embeddings, we evaluate ERMadapters, linear probes, and contrastive adapters on the group-imbalanced BREEDS Living-17training set, and compare their worst-group accuracy to that achieved after training on the defaultgroup-balanced training set.
As hypothesized, we find that ERM adapters (and linear probes) performsignificantly worse after training with the group-imbalanced data.
Meanwhile, the contrastive adaptersare more robust to this setting.
They now outperform the other ERM approaches by 4.0 pp and 2.0pp on worst-group accuracy for CLIP ResNet-50 and ViT-L/14 models respectively.
In contrast tothe results in Table 3, contrastive adapting can outperform ERM training in settings with multipleclasses and subclass group shift.
Notably, from Appendix E.6.2, we saw that the BREEDS Living-17dataset exhibits significant group shifts, and this improvement over ERM occurs when we makethe BREEDS Living-17 dataset harder with group-imbalanced training data.
This is also consistentwith the improved performance of contrastive adapting over ERM on Waterbirds and CelebA, whichexhibit significant group shift (c.f. Table 18) and group imbalance (c.f.
Table 6).
Thus, contrastiveadapting may enable group robust classification in harder settings where ERM training is insufficient.
F Limitations and societal impact
Method limitations.
While in this work, we demonstrated that we can substantially improve thegroup robustness of foundation model classification without any finetuning of the original model,several limitations still exist.
First, this does not imply that we can get desirable performance ingeneral without additional retraining.
To obtain high worst-group performance in general, we areupper-bounded by whether the pretrained embeddings do contain the information needed to classifyall groups.
While our study suggests that in many cases they do carry this information—which cansurprising given that the zero-shot classification with the same embeddings results in poor grouprobustness—in other situations the pretrained embeddings may lack this information.
For example, ifdownstream task data is very different in distribution from the pretraining data, then the pretrainedfoundation model embeddings may not be sufficient to work with.
While more efficient ways toimprove robustness can democratize foundation model use, further finetuning may still be needed.
We also emphasize that our approach is a simple first-step method to improving the group robustnessover existing baseline approaches.
This is motivated by our observation that foundation model zeroshot classification may not be group robust, and that we would like to both (i) improve performanceof these models when we realize they fail in certain aspects, and (ii) do so efficiently, such that fixingtheir failures is not bound by who can conduct costly retraining procedures, and when they can do so.
We are excited for future work and expect further improvements as this thread of how to efficientlyimprove FM performance with limited access ( e.g ., only pretrained embeddings) is further explored.
Societal impact and related limitations.
Finally, we note that it is important to carefully studythe learned biases of foundation models, and to devise appropriate solutions evaluated outside ofjust computational metrics.
Due to their promise of widespread and effective downstream transfer,foundation models may have a particularly strong impact on various parts of society.
Individuals mayget the sense that they can successfully apply these pretrained models to their desired downstreamtasks “out-of-the-box”.
However, doing so also risks applying any learned biases of the model.
Our work raises this issue with respect to group robustness as motivation for our problem setting,and also notes that additional evaluation beyond average accuracy can shed light on the negativequalities of existing models (where zero-shot FM classification may perform very well on average,but very poorly on certain groups, c.f. Figure 6).
However we recognize the limitations of purelycomputational solutions to addressing group performance disparities in society.
We also recognizethe need to better understand foundation models and their potential uses in broader socio-technicalsystems [8].
[Table]
photo of a {}.”
[Table]
picture of a {}
[Table] [Table]
“This is a photo of a {}.”
[Table]
“This is a picture of a {}.”
[Table]
“a {}”
FMoW-WILDS “satellite imagery of a {}.”
“aerial imagery of a {}.”
“satellite photo of a {}.”
“aerial photo of a {}.”
“satellite view of a {}.”
“aerial view of a {}.”
“satellite imagery of the {}.”
“aerial imagery of the {}.”
“satellite photo of the {}.”
“aerial photo of the {}.”
“satellite view of the {}.”
“aerial view of the {}.”
Amazon
[Table]
WILDS “negative”“negativereview”“negative .”
“negativereview.”
“Negative”“Negativereview”“Negative .”
“Negativereview.”
“Thisisanegative review” “This is a negative review.”
“this is a negative review”
CivilComments-WILDS
“This comment is not toxic” “This comment is not toxic.” “this comment is not toxic” “this comment is not toxic.”
“This is not a toxic comment.”
“this is not a toxic comment.” “this is not a toxic comment” “Not toxic.”
“Not toxic.”
“Not toxic” “not toxic” “not
[Table] [Table]