Title: Zero-Shot Robustification of Zero-Shot Models with Auxiliary Foundation Models
Venue: NeurIPS 2023
Accepted: no
Keywords: zero-shot classification

spurious correlations

invariant embedding

foundation model

language model
Abstract: Zero-shot inference is a powerful paradigm that enables the use of large pretrained models for downstream classification tasks without further training. However, these models are vulnerable to inherited biases that can impact their performance. The traditional solution is fine-tuning, but this undermines the key advantage of pretrained models, which is their ability to be used out-of-the-box. We propose RoboShot, a method that improves the robustness of pretrained model embeddings in a fully zero-shot fashion. First, we use zero-shot language models (LMs) to obtain useful insights from task descriptions. These insights are embedded and used to remove harmful and boost useful components in embeddings---without any supervision. Theoretically, we provide a simple and tractable model for biases in zero-shot embeddings and give a result characterizing under what conditions our approach can boost performance. Next, we make pretrained models invariant to spurious features by projecting pre-trained models embeddings on the subspace orthogonal to the spurious feature subspace, which are spanned by the spurious feature descriptions. Empirically, we evaluate RoboShot on nine image and NLP classification tasks and show an average improvement of 15.98% over several zero-shot baselines. Additionally, we demonstrate that RoboShot is compatible with a variety of pretrained and language models.

Abstract

Zero-shot inference is a powerful paradigm that enables the use of large pretrained models for downstream classification tasks without further training. However, these models are vulnerable to inherited biases that can impact their performance. The traditional solution is fine-tuning, but this undermines the key advantage of pretrained models, which is their ability to be used out-of-the-box. We propose ROBOSHOT, a method that improves the robustness of pretrained model embeddings in a fully zero-shot fashion. First, we use zero-shot language models (LMs) to obtain useful insights from task descriptions. These insights are embedded and used to remove harmful and boost useful components in embeddings-without any supervision. Theoretically, we provide a simple and tractable model for biases in zero-shot embeddings and give a result characterizing under what conditions our approach can boost performance. Empirically, we evaluate ROBOSHOT on nine image and NLP classification tasks and show an average improvement of 15.98% over several zero-shot baselines. Additionally, we demonstrate that ROBOSHOT is compatible with a variety of pretrained and language models.

1 Introduction

Zero-shot models are among the most exciting paradigms in machine learning. These models obviate the need for data collection and model training loops by simply asking the model for a prediction on any set of classes. Unfortunately, such models inherit biases or undesirable correlations from their large-scale training data [DLS + 18, TE11]. In a now-canonical example [KSM + 21], they often associate waterbirds with water background . This behavior leads to decreased performance, often exacerbated on rare data slices that break in-distribution correlations.

A growing body of literature [YNPM23, GKG + 22, ZR22] seeks to improve robustness in zero-shot models. While promising, these works require labeled data to train or fine-tune models, and so do not tackle the zero-shot setting. A parallel line of research seeking to debias word embeddings [AZS + , BCZ + 16, DP19, LGPV20] often sidesteps the need for labeled data. Unfortunately, these works often require domain expertise and painstaking manual specification in order to identify particular concepts that embeddings must be invariant to. As a result, out-of-the-box word embedding debiasing methods also cannot be applied to zero-shot robustification.

Can we robustify zero-shot models without (i) labeled data, (ii) training or fine-tuning, or (iii) manual identification? Surprisingly, despite this seemingly impoverished setting, it is often possible to do so. Our key observation is that zero-shot models contain actionable insights that can be exploited to improve themselves or other zero-shot models. These insights are noisy but cheaply available at scale-and can be easily translated into means of refinement for zero-shot representations. These refinements improve performance, particularly on underperforming slices-at nearly no cost.

Task description:

"landbird"

embedding

"Classifywaterbirdvs

1312

landbird"

Auxiliary

Insights

foundation

Useful:beakshape

"landbird"

image

model

Spurious:background

embedding

embedding

23

Insights

embedding

Zero-shot

.81

Landbirdwithwater

model(e.g

CLIP)

.83

1312343

21

Prediction

background

.92

.71

.17

.85

.81

.89

model(e.g.

Zero-shot

CLIP)

.81

83

·15

.12

31

["waterbird","landbird"]

.21

.73

Prediction

background

Landbirdwithwater

.71

Projection

.51

.23

.85

81

.17

.45

89

"waterbird"

["waterbird"，"landbird"]

.21

projected

.73

embedding

Original

embedding

embedding

"waterbird"

embedding

We propose ROBOSHOT, a system that robustifies zero-shot models via auxiliary language models 36 without labels, training, or manual specification . Using just the task description, ROBOSHOT obtains 37 positive and negative insights from a language model (potentially the model to be robustified itself). 38 It uses embeddings of these noisy insights to recover harmful, beneficial , and benign subspaces of 39 zero-shot latent representation spaces. Representations are then modified to neutralize and emphasize 40 their harmful and beneficial components, respectively. 41

Theoretically, we introduce a simple and tractable model to capture and quantify failures in zero-shot 42

models. We provide a result that characterizes the quantity and quality of insights that must be 43

obtained as a function of the severity of harmful correlations. Empirically, ROBOSHOT achieves 44

15.98% improvement across nine image and NLP datasets while offering sufficient versatility to apply 45

to a diverse variety of base models. Most excitingly, in certain cases, it reaches comparable or greater 46

improvements even when compared to fine-tuned models that rely on labeled data. 47

Our contributions include, 48

49

50

51

A simple theoretical model describing zero-shot model failures along with a theoretical analysis of our approach that characterizes the amount of information required for obtaining improvements as a function of the most harmful unwanted correlation,

52

53

54

55

56

57

58

ROBOSHOT, an algorithm that implements our core idea. It extracts insights from foundation models and uses them to improve zero-shot representations,

Extensive experimental evidence on zero-shot language and multimodal models, showing improved worst-group accuracy of 15.98% across nine image and NLP datasets.

2 Related Work

We describe related work in zero-shot model robustness, debiasing embeddings, guiding multi-modal models using language, and using LMs as prior information.

Zero-Shot inference robustness. Improving model robustness to unwanted correlations is heav59 ily studied [SKHL19, ABGLP19, KCJ + 21, KIW22, LHC + 21, LCT + 22]. Some methods require 60 training from scratch and are less practical when applied to large pretrained architectures. Existing 61 approaches to improve robustness post-pretraining predominantly focus on fine-tuning. [YNPM23] 62 detects spurious attribute descriptions and fine-tunes using these descriptions. Specialized contrastive 63 loss is used to fine-tune a pretrained architecture in [GKG + 22] and to train an adapter on the frozen 64 embeddings in [ZR22]. While promising, fine-tuning recreates traditional machine learning pipelines 65 (e.g., labeling, training, etc.), which contradicts the promise of zero-shot models. In contrast, our 66 goal is to avoid any training and any use of labeled data. 67

Debiasing embeddings. A parallel line of work seeks to de-bias text embeddings [AZS + ] 68

[BCZ + 16] [DP19] [LGPV20] and multimodal embeddings [WZS22, BHB + 22, WLW21] by re69

(a)

(b)

S

OText embedding

Projected image embedding

Original image embedding

original embedding

projected embedding

Landbird on land

"Landbird"

-0.2

X

"Land"

-0.3

-0.4

Landbird on water

Waterbird on land

-0.5

"Water"

-0.45 -0.40-0.35-0.30

-0.45 -0.40 -0.35-0.30

xclass =c1

x class = c0

×

c1

"Waterbird"

Waterbird on water

moving subspaces that contain harmful or unwanted concepts. We use a similar procedure as a 70 building block. However, these methods either target specific fixed concepts (such as gender) or rely 71 on concept annotations, which limits their applicability across a wide range of tasks. In contrast, our 72 method automates getting both beneficial and unwanted concepts solely from the task descriptions. 73 An additional difference is that our goal is simply to add robustness at low or zero-cost; we not seek 74 to produce fully-invariant representations as is often desired for word embeddings. 75

Using language to improve visual tasks A large body of work has shown the efficacy of using 76 language to improve performance on vision tasks [RKH + 21, FCS + 13, LCLBC20]. Most relevant 77 are those that focus on robustness, like [PDN + 22], where attention maps using multimodal models 78 (like CLIP) are used as extra supervision to train a downstream image classifier. [YNPM23] uses 79 text descriptions of spurious attributes in a fine-tuning loss to improve robustness against spurious 80 correlations. In contrast to these works, we focus on using textual concepts to improve zero-shot 81 model robustness-without fine-tuning. 82

Language model as prior The basis of our work comes from the observation that language models 83 contain information that can serve as a prior for other learning tasks. [KNST23] finds that LLMs can 84 perform causal reasoning tasks, substantially outperforming existing methods. [CCSE22] explicitly 85 prompts LLMs for task-specific priors, leading to substantial performance improvements in feature 86 selection, reinforcement learning, and causal discovery. Our work shares the spirit of these approaches 87 in using the insights embedded in language models to enhance zero-shot robustness. 88

89

3 RoboShot: Robustifying Zero-shot Models

We are ready to provide our setup and describe the algorithm. 90

3.1 Modeling and setup 91

Suppose that the zero-shot model's latent space contains an (unknown) concept set ; similar notions 92 have been studied frequently in the literature [DKA + ]. For simplicity, we assume that this concept 93 set is given by the orthonormal vectors { z 1 , . . . , z k } . The model's encoder produces, for a particular 94 input a representation x that is a mixture of concepts ∑ i γ i z i , where γ i ≥ 0 are weights. 95

We shall work with the following theoretical model for zero-shot classification. It closely resembles

Algorithm 1: ROBOSHOT

1: Parameters: Input embedding x , class embeddings c 0 , c 1 , harmful insight representations v 1 , . . . , v | S | , helpful insight representations u 1 , . . . , u | R |

2: for j ∈ { 1 , 2 , . . . , | S |} do

3: Reject harmful insight v j : set x ← x -⟨ x, v j ⟩ / ⟨ v j , v j ⟩ v j

4: Renormalize x = x/ ∥ x ∥

5: end for

6: for k ∈ { 1 , 2 , . . . , | R |} do

7: Increase helpful insight u k : set x ← x + ⟨ x, u k ⟩ / ⟨ u k , u k ⟩ u k

8: end for

9: ˆ c = ✶ { x T c 0 < x T c 1 } 10: Returns: Robustified zero-shot prediction ˆ c

Next, we assume that each input representation x can be represented by partitioning the mixture components into three groups,

The same holds for class and insight representations.

Example We illustrate how harmful correlations produce errors on rare slices of data through a standard task setting, Waterbirds [KSM + 21]. In this dataset, the goal is to classify landbirds versus waterbirds , and the background ( land or water ) is spurious. Suppose that we have these terms relate to concepts such that z water = -z land and z waterbird = -z landbird .

Consider a datapoint coming from a rare slice infrequently encountered in the training set. This might be an image of a landbird over water. Its embedding might be x = 0 . 7 z water +0 . 3 z landbird . We may also have that

Then, x T c waterbird = 0 . 1 > x T c landbird = -0 . 1 , so that the prediction is waterbird, and thus incorrect. This is caused by the presence of harmful components in both the class embedding (caused by seeing too many images with water described as waterbirds) and the datapoint embedding (where the water background appears). Thus our goal is to remove harmful components (the z s 's) and boost helpful components (the z r 's). We explain our approach towards doing so next.

3.2 ROBOSHOT: Zeroshot robustification with LLM

We describe ROBOSHOT in Algorithm 1. It uses representations of insights from language models to shape input and class embeddings to remove harmful components and boost helpful ones. Figure 2 is helpful in understanding the intuition behind these procedures. The left part (a) illustrates the effect of ROBOSHOT on a true dataset. Note how unhelpful directions are neutralized while others are boosted. The illustration on the right (b) shows this effect on the waterbirds running example.

Obtaining insight representations from LMs The first question is how to obtain insight representations without training. To do so in a zero-shot way, we use textual descriptions of harmful and helpful concepts by querying language models using only the task description . For example, in the Waterbirds dataset, we use the prompt 'What are the biased/spurious differences between waterbirds and landbirds?'. We list the details of the prompts used in the Appendix. Let s 1 , s 2 be the text insights obtained from the answer (e.g., {' water background ,' ' land background '}). We obtain a spurious insight representation by taking the difference of their embedding

1

1

2

2

v

=

g

g

(

(

s

s

)

)

-

-

g

g

(

(

s

s

)

)

,

the analysis below to multiple classes. We take ∑ i α i z i to be the embedding of a datapoint, while 98 c 0 = ∑ i β i, 0 z i is the embedding of the first class and c 1 = ∑ i β i, 1 z i is that of the second. Finally, 99 we assume that we have access to m answers v 1 , . . . , v m from the queries to the language model. 100 These are given by v j = ∑ i γ i,j z i for j ≤ m . We call these insight representations . Without our 101 approach, the prediction is made by ✶ { ( ∑ i α i z i ) T ( ∑ i β i, 0 z i ) < ( ∑ i α i z i ) T ( ∑ i β i, 1 z i ) } , so that 102 we predict whichever class has higher inner product with the datapoint's embedding. 103 104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

where g is the text encoder of our model.

∥

∥

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

In addition to attempting to discover harmful correlations, we seek to discover helpful components in order to boost their magnitudes past remaining harmful ones (or noise). The procedure is similar. We obtain insight representations using language models. For example, we ask 'What are the true characteristics of waterbirds and landbirds?' and obtain e.g., {' short beak ,' ' long beak '}. The remainder of the procedure is identical to the case of harmful components. Note that since we are seeking to boost (rather than remove) components, it is also possible to fix a multiplicative constant (to be treated as a hyperparameter) for the boosting procedure. That is, we could take x ← x + ν × ⟨ x, u k ⟩ / ⟨ u k , u k ⟩ u k for some ν > 0 . While this is possible if we have access to a labeled set that we can tune ν over, we intentionally avoid doing so to ensure our procedure is truly zero-shot .

Prompting a language model is typically inexpensive, which will enable obtaining multiple insight vectors ˜ v 1 , . . . , ˜ v m . From these, we obtain an orthogonal basis v 1 , . . . , v m separately for harmful and helpful components. Thus we have access to recovered subspaces spanned by such components.

Removing and Boosting Components ROBOSHOT applies simple vector rejection to mitigate or remove harmful components, which is described in lines 2-5 of Algorithm 1. Similarly, it boosts helpful components as described in lines 6-9.

To see the impact of doing so, consider our earlier example. Suppose that v harmful = 0 . 9 z water + 0 . 1 z landbird , and that this is our only harmful insight. Similarly, suppose that we obtain a single helpful insight given by v helpful = 0 . 1 z water + 0 . 9 z landbird . Note that even these insights can be imperfect: they do not uniquely identify what are harmful or helpful concepts, as they have non-zero weights on other components.

We first obtain from removing the harmful component (ignoring normalization for ease of calculation) that

Then, we already we have that x T c waterbird = -0 . 1415 < x T c landbird = 0 . 1415 , so that the correct class is obtained. In other words we have already, from having access to a single insight, neutralized a harmful correlation and corrected what had been an error. Adding in the helpful component further helps. We obtain

This further increases our margin. Note that it is not necessary to fully neutralize (i.e., to be fully invariant to) spurious or harmful components in our embeddings. The only goal is to ensure, as much as possible, that their magnitudes are reduced when compared to helpful components (and to benign components). In the following section, we provide a theoretical model for the magnitudes of such components and characterize the conditions under which it will be possible to correct zero-shot errors. We note that there is a variant of our approach that can also update class embeddings as well.

4 Analysis

Next, we provide an analysis that characterizes under what conditions ROBOSHOT is capable of correcting zero-shot errors. First, we consider the following error model on the weights of the various representations. For all benign representations, we assume that α b , β b , γ b ∼ N (0 , σ 2 benign ) . That is, the magnitudes of benign components are drawn from a Gaussian distribution. The value of σ benign is a function of the amount of data and the training procedure for the zero-shot model.

̸

Next, we assume that the embedding insight v s = ∑ k i =1 γ i,s z i (where 1 ≤ s ≤ S ) satisfies the 172 property that for i = s , γ i,s ∼ N (0 , σ 2 insight ) , while γ s,s is a constant. In other words, the vectors 173 v 1 , . . . , v S spanning the harmful component subspace are well-aligned with genuinely harmful 174 concepts, but are also affected by noise. We seek to understand the interplay between this noise, 175 benign noise, and the coefficients of the other vectors (i.e., helpful components). Let the result of 176 rejecting embedding insights v 1 , . . . , v S be 177

We provide a bound on A s , the coefficient of a targeted harmful concept post-removal. 178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

Theorem 4.1. Under the noise model described above, the post-removal coefficient for harmful concept s satisfies

̸

where k is the number of concepts.

The theorem illustrates how and when the rejection component of ROBOSHOT works-it scales down harmful coefficients at a rate inversely proportional to the harmful coefficients of the insight embeddings. As we would hope, when insight embeddings have larger coefficients for harmful vectors (i.e., are more precise in specifying terms that are not useful), ROBOSHOT yields better outcomes. In addition, we observe that the harmful coefficients decrease when the insight embeddings have less noise. In fact, we have that lim σ insight → 0 A s = 0 -the case of perfectly identifying harmful concepts. In the Appendix, we present additional theoretical results for control of helpful coefficients along with a combined result.

5 Experimental Results

This section evaluates the following claims about ROBOSHOT:

Improving multi-modal models (Section 5.1) : ROBOSHOT improves zero-shot classification robustness of various multi-modal models, even outperforming prompting techniques that include spurious insight descriptions (which we do not have access to) in the label prompts.

Improving language models (Section 5.2) : ROBOSHOT improves zero-shot robustness when using language model embeddings for text zero-shot classification.

Extracting concepts from LM with varying capacities (Section 5.3) : ROBOSHOT can extract insights from language models with varying capacities. Improvements persist with weaker LMs.

Ablations (Section 5.4) ROBOSHOT benefits from both removing harmful and boosting helpful representations (line 3 and line 7 in ROBOSHOT Algorithm 1).

Metrics and how to interpret the results. We use three metrics: average accuracy % (AVG), worst-group accuracy % (WG), and the gap between the two (Gap). While a model that relies on harmful correlations may achieve high AVG when such correlations are present in the majority of the test data, it may fail in settings where the correlation is absent. A robust model should have high AVG and WG, with a small gap between them .

Baselines We compare against the following sets of baselines:

Multimodal baselines : We compare against: (i) vanilla zero-shot classification ( ZS ) and (ii) zero-shot classification with group information ( Group Prompt ZS ). We do so across a variety of models: CLIP (ViT-B-32 and ViT-L-14) [RKH + 21], ALIGN [JYX + 21], and AltCLIP [CLZ + 22]. Group Prompt ZS assumes access to spurious or harmful insight annotations and includes them in the label prompt. For instance, the label prompts for waterbirds dataset become [ waterbird with water background , waterbird with land background , landbird with water background , landbird with land background ]. We only report Group Prompt ZS results on datasets where spurious insight annotations are available.

Language model baselines : We compare against zero-shot classification using multiple language model embeddings, including BERT [RG19] and Ada [NXP + 22] ( ZS ).

5.1 Improving multi-modal models

Setup. We experimented on five binary and multi-class datasets with spurious correlations and distribution shifts, coming from a variety of domains: Waterbirds [SKHL19], CelebA [LLWT15], CXR14 [WPL + 17], PACS [LYSH17], and VLCS [FXR13]. We use the default test splits of all datasets. Dataset details are provided in the appendix. For CXR14, we use BiomedCLIP [ZXU + 23],

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

(a)

(b)

ALIGNWaterbirds

CLIPWaterbirds

ALIGNWaterbirds

CLIPWaterbirds

0.3

★

★

0.1

1

0.2

★

2

★

0.0-

0.1

-0.1

0

-0.1

0.0

★

0

★

★

☆

★

长长

★

★

-0.2

赛

-2

★

★

☆

0.00

0.25

0.50

0.75

0.0

0.5

1.0

0.00

0.25

0.50

0.75

-2

0

2

4

×

c1

0

x original

xprojected

×

c1

★

vi

★

ui

which is a variant of CLIP finetuned on biomedical images and articles. All experiments are conducted using frozen pretrained models.

Results. Table 1 shows that ROBOSHOT significantly improves the worst group performance (WG) and maintains (and sometimes also improves) the overall average (AVG) without any auxiliary information (in contrast to Group Prompt, which requires access to spurious insight annotation).

Improved robustness nearly across-the-board suggests that both the insights extracted from LMs and the representation modifications are useful. We also provide insights insights into the case where our method does not improve the baseline (ALIGN model on Waterbirds) in Fig. 3. In Fig. 3a, we visualize the original and projected input embeddings ( x in green and red points, respectively), and the label embeddings ( c 0 and c 1 ). Fig. 3a (left) shows the embeddings from the ALIGN model. We observe that the projected embeddings (red) still lie within the original embedding space, even with reduced variance. In contrast, when examining the CLIP model embeddings (Figure 3a (right)), we observe that the projected embeddings are significantly distant from the original ones. Unsurprisingly, Figure 3b (left) reveals that v j and u k (harmful and helpful insight embeddings in black and blue stars, respectively) are not distinguishable in the text embedding space of ALIGN, collapsing the input embeddings after ROBOSHOT is applied.

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

5.2 Improving language models

Setup. We experimented on four text classification datasets: CivilComments-WILDS [BDS + 19, KSM + 21], HateXplain [MSY + 21], Amazon-WILDS [NLM19, KSM + 21] and Gender Bias classification dataset [DFW + 20, MFB + 17]. We use the default test splits of all datasets. In text experiments, the distinctions between harmful and helpful insights are less clear than for images. For this reason, we only use harmful vector rejection (line 3 in ROBOSHOT) in text experiments. CivilComments and HateXplain are toxic classification datasets with unwanted correlation between toxicity labels and mentions of demographics (e.g., male, female, mentions of religions). The datasets are annotated with demographic mentions of each text, and we directly use them to construct v j . For Amazon and Gender Bias datasets, we query LMs with task descriptions. All experiments are conducted using frozen pretrained models.

Results. Table 2 shows that ROBOSHOT also improves zero-shot text classification in text datasets, as shown by our consistent boost over the baselines across all datasets.

5.3 Extracting concepts from LMs with varying capacities

Setup. We use LMs with different capacities: ChatGPT [OWJ + 22], Flan-T5 [CHL + 22], GPT2 [RWC + 19], and LLaMA [TLI + 23], to get harmful and helpful features insights ( v j and u k ).

Results. Table 3 shows that ROBOSHOT can get insights on v j and u k from LMs of various capacities and improves zero-shot performance. Even though the the LM capacity correlates with the zero-shot performance, ROBOSHOT with weaker LMs still outperforms zero-shot (ZS) baseline.

5.4 Ablations

Setup. We run ROBOSHOT with only harmful component mitigation (reject v j : ROBOSHOT line 3), only boosting helpful vectors (increase u k : ROBOSHOT line 7), and both.

Results. The combination of both projections often achieves the best performance, as shown in Table 4. Figure 4 provides insights into the impact of each projection. Rejecting v j reduces variance in one direction, while increasing u k amplifies variance in the orthogonal direction. When both projections are applied, they create a balanced mixture. We note that when doing both projections does not

original embedding

reject vi

increase uk

both

-0.2

X

X

X

-0.3

-0.4

-0.5

-0.4

-0.3

-0.4

-0.3

-0.4

-0.3

-0.4

-0.3

x class = c1

x class = c0

YO

×

Y1

improve the baseline, using only u k or v j still outperforms the baseline. For instance, the ALIGN 265 model in the Waterbirds dataset achieves the best performance with only u k projection. This suggests 266 that in certain cases, harmful and helpful concepts are intertwined in the embedding space, and using 267 just one projection can be beneficial. We leave further investigation to future work. 268

269

270

271

272

273

274

275

276

277

278

279

6 Conclusion

We introduced ROBOSHOT, a fine-tuning-free system that robustifies zero-shot pretrained models in a truly zero-shot way. Theoretically, we characterized the quantities required to obtain improvements over vanilla zero-shot classification. Empirically, we found that ROBOSHOT improves both multimodal and language model zero-shot performance, has sufficient versatility to apply to various base models, and can use insights from less powerful language models.

References

[ABGLP19] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893 , 2019.

[AZS + ] Prince Osei Aboagye, Yan Zheng, Jack Shunn, Chin-Chia Michael Yeh, Junpeng Wang, Zhongfang Zhuang, Huiyuan Chen, Liang Wang, Wei Zhang, and Jeff Phillips.

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

Interpretable debiasing of vectorized language representations with iterative orthogonalization. In The Eleventh International Conference on Learning Representations .

[BCZ + 16] Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 29. Curran Associates, Inc., 2016.

[BDS + 19] Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference , pages 491-500, 2019.

[BHB + 22] Hugo Berg, Siobhan Mackenzie Hall, Yash Bhalgat, Wonsuk Yang, Hannah Rose Kirk, Aleksandar Shtedritski, and Max Bain. A prompt array keeps the bias away: Debiasing vision-language models with adversarial learning. arXiv preprint arXiv:2203.11933 , 2022.

[CCSE22] Kristy Choi, Chris Cundy, Sanjari Srivastava, and Stefano Ermon. Lmpriors: Pre-trained language models as task-specific priors. arXiv preprint arXiv:2210.12530 , 2022.

[CHL + 22] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instructionfinetuned language models. arXiv preprint arXiv:2210.11416 , 2022.

[CLZ + 22] Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Fulong Ye, Qinghong Yang, and Ledell Wu. Altclip: Altering the language encoder in clip for extended language capabilities. arXiv preprint arXiv:2211.06679 , 2022.

[DFW + 20] Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. Multi-dimensional gender bias classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages 314-331, Online, November 2020. Association for Computational Linguistics.

[DKA + ] Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, Nadir Durrani, Jia Xu, and Hassan Sajjad. Discovering latent concepts learned in bert. In International Conference on Learning Representations .

[DLS + 18] Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Measuring and mitigating unintended bias in text classification. 2018.

[DP19] Sunipa Dev and Jeff Phillips. Attenuating bias in word vectors. In The 22nd International Conference on Artificial Intelligence and Statistics , pages 879-887. PMLR, 2019.

[FCS + 13] Andrea Frome, Greg S Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc'Aurelio Ranzato, and Tomas Mikolov. Devise: A deep visual-semantic embedding model. Advances in neural information processing systems , 26, 2013.

[FXR13] Chen Fang, Ye Xu, and Daniel N Rockmore. Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In Proceedings of the IEEE International Conference on Computer Vision , pages 1657-1664, 2013.

[GKG + 22] Sachin Goyal, Ananya Kumar, Sankalp Garg, Zico Kolter, and Aditi Raghunathan. Finetune like you pretrain: Improved finetuning of zero-shot vision models. arXiv preprint arXiv:2212.00638 , 2022.

[JYX + 21] Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning , pages 4904-4916. PMLR, 2021.

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

[KCJ + 21] David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distribution generalization via risk extrapolation (rex). In International Conference on Machine Learning , pages 5815-5826. PMLR, 2021.

[KIW22] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. arXiv preprint arXiv:2204.02937 , 2022.

[KNST23] Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050 , 2023.

[KSM + 21] Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the-wild distribution shifts. In International Conference on Machine Learning , pages 5637-5664. PMLR, 2021.

[LCLBC20] Yannick Le Cacheux, Hervé Le Borgne, and Michel Crucianu. Using sentences as semantic representations in large scale zero-shot learning. In Computer Vision-ECCV 2020 Workshops: Glasgow, UK, August 23-28, 2020, Proceedings, Part I 16 , pages 641-645. Springer, 2020.

[LCT + 22] Yoonho Lee, Annie S Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. arXiv preprint arXiv:2210.11466 , 2022.

[LGPV20] Anne Lauscher, Goran Glavaš, Simone Paolo Ponzetto, and Ivan Vuli´ c. A general framework for implicit and explicit debiasing of distributional word vector spaces. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 8131-8138, 2020.

[LHC + 21] Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning , volume 139 of Proceedings of Machine Learning Research , pages 6781-6792. PMLR, 18-24 Jul 2021.

[LLWT15] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision , pages 3730-3738, 2015.

[LYSH17] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE international conference on computer vision , pages 5542-5550, 2017.

[MFB + 17] Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. ParlAI: A dialog research software platform. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations , pages 79-84, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.

[MSY + 21] Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 35, pages 14867-14875, 2021.

[NLM19] Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) , pages 188-197, 2019.

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

[NXP + 22] Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005 , 2022.

[OWJ + 22] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35:27730-27744, 2022.

[PDN + 22] Suzanne Petryk, Lisa Dunlap, Keyan Nasseri, Joseph Gonzalez, Trevor Darrell, and Anna Rohrbach. On guiding visual attention with language specification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 18092-18102, 2022.

[RG19] Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 , 2019.

[RKH + 21] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PMLR, 2021.

[RWC + 19] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog , 1(8):9, 2019.

[SKHL19] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 , 2019.

[TE11] Antonio Torralba and Alexei A. Efros. Unbiased look at dataset bias. In CVPR 2011 , pages 1521-1528, 2011.

[TLI + 23] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 , 2023.

[WLW21] Jialu Wang, Yang Liu, and Xin Eric Wang. Are gender-neutral queries really genderneutral? mitigating gender bias in image search. arXiv preprint arXiv:2109.05433 , 2021.

[WPL + 17] Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2097-2106, 2017.

[WZS22] Junyang Wang, Yi Zhang, and Jitao Sang. Fairclip: Social bias elimination based on attribute prototype learning and representation neutralization. arXiv preprint arXiv:2210.14562 , 2022.

[YNPM23] Yu Yang, Besmira Nushi, Hamid Palangi, and Baharan Mirzasoleiman. Mitigating spurious correlations in multi-modal models during fine-tuning. arXiv preprint arXiv:2304.03916 , 2023.

[ZR22] Michael Zhang and Christopher Ré. Contrastive adapters for foundation model group robustness. arXiv preprint arXiv:2207.07180 , 2022.

[ZXU + 23] Sheng Zhang, Yanbo Xu, Naoto Usuyama, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, Cliff Wong, Matthew Lungren, Tristan Naumann, and Hoifung Poon. Large-scale domain-specific pretraining for biomedical visionlanguage processing, 2023.

Peer reviews:
comments: SUMMARY:
This paper introduces RoboShot, a method that improves the robustness of pretrained model embeddings in a fully zero-shot manner. The key idea is to leverage insights obtained from language models based on task descriptions. These insights are used to modify the embeddings, removing harmful components and enhancing useful ones, without the need for supervision. Technically, the method ensures invariance to spurious features by projecting pretrained model embeddings onto the subspace orthogonal to the subspace spanned by spurious feature descriptions. Experiments demonstrate that RoboShot improves multi-modal and language model zero-shot performance.

STRENGTHS:
1. **Novel and useful setting:** The setting of improving the robustness of pretrained model embeddings with task description is novel. RoboShot offers a unique approach that preserves the out-of-the-box usability of pretrained models, which is a key advantage.

2. **Extensive experiments and analyses:** The authors demonstrated the efficacy of the proposed method and setting with extensive experiments and analyses, in terms of both datasets and settings. 

3. The paper is well-written.

WEAKNESSES:
1. **Limitation of method:** The robustification relies on the insights provided by language models. However, if the language model does not identify the potential failure cases of the model, the method cannot remedy it. 

2. **Gender bias:** Some experiments are targeted at gender bias. It's better to discuss the scope of evaluation here, e.g., what genders are considered and what biases remain unresolved.

QUESTIONS:
1. What is the limitation of the proposed method, and when does it fail?

LIMITATIONS:
Limitation needs to be elaborated on.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-06-11

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents an innovative approach to enhance zero-shot classification inference without the need for fine-tuning pre-trained models. The authors introduce a method that partitions input embeddings into three components: harmful, helpful, and benign. By leveraging task descriptions and querying language models with harmful and helpful prompts, they successfully extract harmful and helpful components. This leads to the removal of harmful components and a boost in the helpful ones, ultimately improving robustness in zero-shot classification. The proposed method demonstrates promise and contributes to the field of zero-shot classification without extensive model finetuning.\n\nSTRENGTHS:\nThe paper is well-structured and easily understandable, with a strong and compelling motivation. As model sizes continue to increase, fine-tuning LLMs becomes increasingly expensive. This paper presents a compelling alternative by enhancing zero-shot performance while retaining the power of LLMs without the need for fine-tuning.  The innovative method of partitioning embeddings into three concepts and leveraging task descriptions and LLMs to strengthen or weaken them is intriguing and holds promise for embedding-based zero-shot text classification.\n\nWEAKNESSES:\n1. The proposed method is primarily applicable to embedding-based zero-shot classification approaches, while prompt-based methods like ChatGPT3.5/4 have gained popularity recently. Prompt-based methods allow humans to directly query language models based on downstream task knowledge, resulting in impressive performance. Although the proposed method is interesting for embedding-based zero-shot classification, its impact may be limited due to the current research trend.\n\n2. Considering the above point, it would be beneficial if the authors compare their method to a more reasonable baseline, such as asking ChatGPT about predictions. This approach could be employed, for instance, by asking ChatGPT to identify if a given text contains gender bias. While ChatGPT's performance might not extend to image-based tasks, applying this baseline to text datasets would provide valuable insights. If ChatGPT achieves accurate predictions, it questions the necessity of an embedding-based zero-shot text classification method.\n\n3. Although the method exhibits significant improvement in worst group performance (WG), it would strengthen the findings if the overall average performance also demonstrated improvement. It is worth noting that in some datasets, the average performance did not improve. This implies that the proposed method sacrifices performance for some classes to achieve better results in the WG. Ensuring a balance between overall average performance and WG improvement would bolster the method's effectiveness.\n\nOverall, I am inclined to accept the paper if all weaknesses are properly addressed but I am also not strongly against rejecting the paper.\n\nQUESTIONS:\n1. The paper mentions that the average performance of some datasets is not improved. Could you please explain the possible reasons for this lack of improvement?\n\n2. I'm curious about the performance of zero-shot text classification using ChatGPT. Can you provide any insights or comparative results on the performance achieved with ChatGPT for zero-shot text classification tasks?\n\n3. In the paper, questions are posed to LLMs to extract helpful or harmful insights. Could you elaborate on how these questions are designed? Does it involve a significant amount of human decision-making and prior knowledge about the datasets?\n\nLIMITATIONS:\nPlease refer to weaknesses.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel approach called ROBOSHOT that aims to enhance the robustness of pre-trained models for zero-shot classification tasks. The key idea is to leverage task-specific queries to prompt the large language model (LLM) to generate textual insights about the task, which can be classified as helpful or harmful concepts. These textual insights are then used to obtain \"insight representations\" based on the corresponding encoded embeddings, which are utilized to calibrate the vector of input representation to predict the class. The proposed method is evaluated through experiments on zero-shot image and text classification tasks. The paper also includes a theoretical analysis of the bound of the coefficient of targeted harmful concept post-removal.\n\nSTRENGTHS:\n1. The motivation of this paper is clear. The problem of \"robustifying zero-shot models without labels, training, or manual specification\" is challenging and interesting. \n2. The paper proposes a simple method to calibrate the input embedding to make predictions.\n3. The paper is generally easy to read and easy to follow. The settings of experiments in this paper are clear.\n4. The theoretical proof of the coefficient bound of the targeted harmful concept is a plus.\n\nWEAKNESSES:\n1. The paper aims to robustify zero-shot models without labels, training, or manual specification. However, the proposed method still requires the use of manually-designed helpful/harmful queries (shown in Table 6&7) to query the LLM. \n2. The paper did not explore the impact of using different prompts to query textual insights. Given that the experiments were conducted on small pre-trained models such as BERT, which are less robust than LLMs, it is unclear whether the resulting different text insights would significantly affect the model's performance.\n3. The paper employs LLMs (e.g., chatgpt/LLAMA) to generate textual insights for calibrating the input embedding of a small pre-trained model for prediction. It raises the question of why not directly prompt the LLM for zero-shot text classification if it already possesses sufficient knowledge about each class. The paper would benefit from including such a baseline comparison.\n4. The paper did not include and compare some relevant works, such as [1] [2], which calibrate the logits of predictions for zero-shot/few-shot text classification. \n5. Table 1 indicates that the proposed method does not improve the ALIGN model on the Waterbirds dataset, but it does improve ALIGN's performance on CelebA, VLCS, and CXR14. The paper attributes this discrepancy to the harmful and helpful insight embeddings of Waterbirds being indistinguishable in the text embedding space of ALIGN. However, it is unclear how to determine when the ROBOSHOT method is applicable to different combinations of models and datasets. The paper could benefit from providing some metrics to predict the conditions under which ROBOSHOT is effective.\n\n[1]Surface Form Competition: Why the Highest Probability Answer Isn’t Always Right\n\n[2]Calibrate Before Use: Improving Few-Shot Performance of Language Models\n\nQUESTIONS:\n1. How can we determine when ROBOSHOT is suitable for different models and datasets?\n2. Which dataset and model were used to generate the results presented in Figure 4, and what textual insights correspond to the \"reject\" and \"increase\" categories? Are there additional experiments or analyses that support the consistency of the proposed method's effectiveness with the analysis of the latent space?\n3. Regarding lines 127-130, are the concise textual insights generated by the LLM manually determined by humans, or is there an automated process for deciding them? Given that the LLM is typically used to produce complete sentences, how is it adapted to generate these short and concise insights?\n\nLIMITATIONS:\nplease refer to weakness point 5", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe main objective of this research is to enhance the robustness of image/text classification by taking into account the relationships between labels. The authors utilize Large Language Model (LLM) to incorporate prior knowledge about these labels. They also propose methods to amplify useful features and eliminate harmful features induced from these labels.\n\nIn contrast, prior work require manual identification of biased features for debiasing, whereas this work leverages LLM to automatically suggest such features, thereby reducing the need for manual effort.\n\nSTRENGTHS:\n- The paper is well written and can be easily understood.\n- Using LLM to propose useful and spurious features is somewhat novel.\n\nWEAKNESSES:\n- This work aims to incorporate the knowledge of LLM into image classification. However, the method used, which is based on CLIP, does not seem to fully utilize the knowledge from LLM. The author's approach involves generating insight descriptions to establish a connection between LLM and CLIP's text encoder.  A more intuitive alternative would be to link CLIP's image encoder directly with LLM through an adapter, similar to the Flamingo (https://arxiv.org/abs/2204.14198) and LLaVA (https://arxiv.org/abs/2304.08485) methods. By doing so, the knowledge from LLM can be more comprehensively utilized, rather than solely the generated insight descriptions.\n- The subspace-based debiasing methods have already been used under various contexts as also mentioned in the paper, which cannot be viewed as novel to some extent.\n\nQUESTIONS:\n- The multimodal baseline is good, how about a similar baseline that uses LLM to propose insight descriptions instead of using ground-truth? This gives better insights in comparing the benefit of injecting features in the image embedding side (e.g., by feature projection as introduced in this work) or the label side (e.g., by simply enriching the description).\n- How about the proposed method benefits architectures like LLaVA and Flamingo?\n\nLIMITATIONS:\nThe limitations part is missing in the paper.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-25"}