Title: LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images
Venue: NeurIPS 2023
Accepted: yes
Keywords: image classification

robustness

guided diffusion models

counterfactuals
Abstract: We propose an automated algorithm to stress-test a trained visual model by generating language-guided counterfactual test images (LANCE). Our method leverages recent progress in large language modeling and text-based image editing to augment an IID test set with a suite of diverse, realistic, and challenging test images without altering model weights. We benchmark the performance of a diverse set of pre-trained models on our generated data and observe significant and consistent performance drops. We further analyze model sensitivity across different types of edits, and demonstrate its applicability at surfacing previously unknown class-level model biases in ImageNet. Code is available at https://github.com/virajprabhu/lance.

LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images

Abstract

We propose an automated algorithm to stress-test a trained visual model by generating language-guided counterfactual test images (LANCE). Our method leverages recent progress in large language modeling and text-based image editing to augment an IID test set with a suite of diverse, realistic, and challenging test images without altering model weights. We benchmark the performance of a diverse set of pretrained models on our generated data and observe significant and consistent performance drops. We further analyze model sensitivity across different types of edits, and demonstrate its applicability at surfacing previously unknown class-level model biases in ImageNet. Code: https://github.com/virajprabhu/lance.

1 Introduction

Test image

(Label: Dog Sled)

ResNet-50

Husky

→

Husky

Labradoodle

→

Husky

Pomeranian

→

'Huskies pulling a dog sled'

27%

10%

Dog Sled

Eskimo Dog

3%

2%

Irish terrier

Irish wolfhound

Ibizan hound

10%

6%

Pomeranian

Chihuahua

0.12%

0.11%

53%

82%

99.7%

64%

As deep visual models become ubiquitous in high-stakes applications, robust stress-testing assumes paramount importance. However, the traditional paradigm of evaluating performance on largescale [2-4] IID test sets does not adequately vet models for deployment in the wild. First, such models are typically evaluated via aggregate metrics such as accuracy, IoU, or average precision [5], which treat all test samples equivalently and do not distinguish between error types. Further, such test sets do not adequately capture the 'long-tail' [6] of the data distribution: rare concepts, unseen concepts, and the (combinatorial explosion) of their compositions [7].

To address this, a considerable number of recent efforts have sought to develop realistic benchmarks for out-of-distribution (OOD) evaluation [8-13]. However, while useful, such OOD benchmarks are typically static across models and time, rather than being curated to probe a specific instance of a trained model, which diminishes their utility.

In this work, we eschew the traditional paradigm of static test sets and instead generate dynamic test suites . Specifically, we propose a technique that automatically generates a test suite of challenging but realistic counterfactual [14] examples to stress-test a given visual model. Our main insight is that while the scope of possible visual variation is vast, language can serve as a concise intermediate scaffold that captures salient visual features while abstracting away irrelevant detail. Further, while it is extremely challenging to run counterfactual queries on images directly, intervening on a discrete text representation is considerably easier. We call counterfactuals generated by our method Languageguided Counterfactual Images (LANCE).

Our method leverages pretrained foundation models for text-to-image synthesis [15-17] and large language modeling (LLM) [18], in combination with recent progress in text-based image editing [19, 20], to generate realistic counterfactual examples. It operates as follows: Given a test image and its ground truth label, we generate a caption conditioned on its label using an image captioning model [21]. We then use an LLM that has been fine-tuned for text editing [22] to generate realistic perturbations to a single concept at a time - caption subject, object, background, adjectives, or image domain (while leaving words corresponding to the ground truth category unchanged). Finally, we use a guided-diffusion model [19] with null-text inversion [20] to generate an edited counterfactual image conditioned on the original image and the perturbed prompt. We obtain a prediction from the trained model on both the original image and the generated counterfactual and compute the average drop in accuracy as a measure of the model's sensitivity to the given attribute.

We benchmark the performance of diverse models pretrained on ImageNet on our generated images, and find that performance drops more consistently and significantly than baseline approaches. Next, we demonstrate applications of our approach: in comparing the relative sensitivity of models to perturbations of different types, and in deriving actionable insights by isolating consistent class-level interventions that change model predictions.

2 Related Work

Diagnosing deep visual models. Several recent works have focused on the discovery of systematic failure modes in trained neural networks. Early works propose human-in-the-loop approaches based on annotating spurious features learned by sparse linear layers [23], adversarially robust models [24], and discovering a hyperplane corresponding to potentially biased attribute [25]. Some prior work also proposes fully automated techniques: by learning decision trees over misclassified instance features from an adversarially robust model [26] or targeted data collection [27]. Several recent approaches to this problem leverage multi-modal CLIP [28] embeddings, via error-aware mixture modeling [29], identifying failure directions in latent space [30] or learning the task directly on top of CLIP embeddings [31]. However, these works rationalize failures but don't close the loop by evaluating the predicted rationale. Further, shortcut learning [32] of spurious correlations can also lead to success but for the wrong reasons , but prior work only studies failure cases. In contrast, we propose a method that generates visual counterfactuals that can surface a diverse range of model biases that bidirectionally influence performance.

Image Editing with generative models. Using generative models to edit images has seen considerable work. Early efforts focused on specialized editing such as style transfer [33] or translating from one domain to another [34]. More recent work has performed editing in latent space of a model like StyleGAN [35-39]. Recently, pretrained text-to-image diffusion models have become the tool of choice for image editing [19, 20, 17]. While superior to GAN's at image synthesis [40], targeted editing using such models that modifies only a specific visual attribute while keeping the rest unchanged is non-trivial. Recent work has presented techniques based on prompt-to-prompt tuning [19], by targeted insertion of attention maps from cross-attention layers during the diffusion process. Follow-up work generalizes this approach to real images, with additional null-text inversion [20] or delta denoising scoring functions [41], which enables instruction-based editing of images [42] and 3D scenes [43]. We leverage this technique of prompt-to-prompt tuning with null-text inversion but rather than generic image editing focus on generating challenging counterfactual examples.

Probing discriminative networks with generated counterfactuals. Several works have attempted to use generative models to obtain explanations from visual models. A popular line of work generates minimal image perturbations that flip a model's prediction [44-47] ( e.g. changing lip curvature to alter the prediction of a 'smiling' classifier). Our work shares a similar goal but with a crucial distinction:

Model

Model

'howler

monkey'

(98% confidence)

'gorilla'

(75% confidence)

Trained Model, Test Set

a black monkey

sitting on top of

a tree with

green leaves

a black monkey

sitting on top of

a tree with

orange leaves

invert

Frozen

Learned

Δp y

!"

x → x′

= |p y

!"

x

-p y

!"

x

#

|

Editor

(DDPM)

Perturber

(LLM)

Captioner

LANCE: Language-guided Counterfactual Image Generation

we seek to discover minimal perturbations to parts of the image not directly relevant to the task at hand ( e.g. does changing hair color or perceived gender alter the smiling classifier's prediction?).

Closest to our work is Luo et al. [48], which determines model sensitivity to a set of user-defined text attributes by tuning a weighted combination of edit vectors in StyleGAN [35] latent space so as to flip the model prediction while maintaining global structure and semantics via additional losses. However, this method requires the user to enumerate all possible attributes of interest, may not generalize to more complex datasets, and requires optimizing several losses in conjunction. Our work also shares similar motivations as Li et al. [49], which uses diffusion models to generate ImageNet-E(diting), a robustness benchmark with varying backgrounds, object sizes, position, and directions. However, their benchmark is static across models, requires object masks per image to generate, and measures robustness to a constrained set of attribute changes. Our work is also related to Wiles et al. [50] which generates cluster-based error rationales, Vendrow et al. [51], which diagnoses failures by generating near-counterfactuals from human-generated text variations, and Dunlap et al. [52], which performs diffusion-based data augmentation. In contrast, we study bias discovery rather than mitigation, deriving causal insights into failures using a targeted editing algorithm, and support stress-testing across an unconstrained, automatically discovered set of attributes.

3 Language-guided Counterfactual Image Generation (LANCE)

Overview. We introduce LANCE, our algorithm for generating language-guided counterfactual images to stress-test a given visual model. Our key insight is to use language as a structured discrete scaffold to perform interventions on the model. LANCE does this by perturbing a text representation (caption) of an image to conditionally generate a counterfactual image (see Fig 2).

Perturbed Captions. We first use a pre-trained captioner (BLIP-2 [21]), to produce a text description for a given test image. We use beam search decoding with a minimum caption length of 20 words and a high repetition penalty to encourage descriptive captions with low redundancy (see Fig. 5 for examples). Our algorithm then edits the generated caption using a language model to generate variations that only change a single aspect at a time.

Edited Images. Next, we use a text-to-image latent diffusion model to generate a new image that reflects the text edit while remaining faithful to the original image in every other respect. We repeat this process for multiple perturbations to generate a challenging test set.

Sensitivity Analysis. Finally, we ascertain model sensitivity to different factors of variation by reporting the change in the model's accuracy over the corresponding counterfactual test set.

Next, we detail each step, beginning with describing the factors of visual variation that we stress-test against and the strategy we use to train a structured caption perturbed. Then, we describe our image

editing methodology and the various checks and balances we insert to ensure that our end-to-end approach yields challenging yet realistic test examples.

3.1 Training a Structured Caption Perturber

To perform meaningful interventions, we train a structured caption perturber that generates targeted caption edits. Motivated by prior work in finetuning instruction-following large language models [22](LLMs), we seek to train a structured perturber LLM that can generate diverse and realistic caption edits that capture the long-tail of visual variation while still representing realistic scenarios.

Selecting factors of variation. To train our perturber model, we select five factors of visual variation against which we will stress-test. While the space of potential factors of visual variation is vast, image captions can constrain this search space by capturing salient visual concepts while abstracting away irrelevant detail. Concretely, we measure resilience to the following factors of variation:

Subject. Modifications to the subject of an image caption (e.g., a {man, dog, cat, ..., horse,} ) can stress-test a model's ability to recognize the ground truth category when it co-occurs with other subjects, including subjects that co-occur rarely (or never) with the ground truth in the training data.

Object. Similarly, modifications to the object of an image caption (e.g., a {table, chair, ..., bed} ) can stress-test a model's resilience to novel or unseen co-occurring concepts.

Background. Modifications to the context of an image caption (e.g., a {kitchen, bedroom, ..., living room} ) can stress-test a model's ability to generalize to different scenes, including diverse backgrounds and weather conditions.

Adjective. Modifications to the adjective of an image caption (e.g., a {red, blue, ..., green} ) can stress-test a model's ability to generalize to visual variations captured by object attributes.

Data domain. Modifications to the data domain of an image caption (e.g., a {painting, sketch, ..., sculpture} ) can stress-test a model's ability to generalize to different data distributions.

While by no means exhaustive, these perturbation types capture a large and representative set of visual variations. Further, we note that our approach is agnostic to this choice and can easily be extended to accommodate additional factors (e.g., camera angle, lighting, etc.). Having defined our perturbation set, we proceed to collect a dataset to train our structured perturbation model.

Dataset collection. We use GPT-3.5 turbo [18] to programmatically collect a small dataset of 0.6k caption perturbations spanning the aforementioned types. For instance, to edit an 'adjective', we prompt the model with: Generate all possible variations of the provided sentence by only adding or altering a single adjective or attribute. (all prompts in appendix). We find that even with zeroshot prompting, we are able to acquire caption perturbations that modify only the desired factor of variation. We follow this procedure to generate perturbations for randomly selected captions from the MSCOCO dataset [4]. Table. 1 ( top ) highlights example inputs and perturbations. As seen, the dataset contains both simple edits ( e.g. , a bicycle → {red, blue, ..., green} bicycle ) as well as more complex ones ( e.g. , a bicycle → bicycle parked outside a coffee shop ), without requiring any human supervision whatsoever.

Model training. Next, we finetune an LLM on the dataset described above to perform targeted caption editing: specifically, we use a LLAMA-7B [53] LLM and perform LoRA (Low-Rank Adaptation) finetuning [54]: the original LLAMA-7B model is kept frozen, while the change in weight matrices W ∈ R d × k of the self-attention modules post-adaptation is restricted to have a low-rank decomposition W ft = W pt +∆ W = W pt + AB , where A ∈ R d × r and B ∈ R r × k . Here the rank r is kept low, resulting in very few trainable parameters in A and B .

We use this finetuned model to perturb generated captions from ImageNet [2]. Table. 1 ( top ) shows examples of perturbations generated by this model. As seen, it is able to generate sensible and diverse edits corresponding to the edit type without changing the remainder of the caption.

Caption editing: Checks and balances. An undesirable type of edit would modify a word corresponding to the ground truth category label of the image. For instance, if the ground truth label of an image is 'dog', we do not want the model to generate a perturbation that changes the word 'dog' to 'cat'. To address this, we simply filter our caption edits wherein the edit is semantically similar (measured using a sentence BERT model [55]) to the ground truth category. Similarly, we also filter our captions where the edit is semantically similar to the original word or phrase being modified.

3.2 Counterfactual Image Generation

Having generated image captions and their perturbations, we proceed to generate counterfactual images conditioned on the original image and edited caption using a text-to-image latent diffusion model, specifically Stable Diffusion [16]. However, despite its remarkable ability at generating highquality text-conditioned images, targeted editing using such models has historically been challenging, as changing even a single word in the text prompt could dramatically change the generated output image. Naturally, this is undesirable for our task, as we want to generate counterfactual images that are as similar as possible to the original image, while still reflecting the caption edit.

To address this, we leverage the recently proposed prompt-to-prompt [19] 1 image editing technique, which performs targeted injection of cross-attention maps that correspond to the caption edit for a subset of the denoising diffusion process. However, prompt-to-prompt is designed for generated images, and applying it to real images requires accurate image inversion to latent space.

We employ the recent null-text inversion technique proposed by Mokady et al. [20]. Concretely, for input image x with caption c , we perform DDIM inversion by setting the initial latent vector z 0 to the encoding of the image to be inverted x , and run the diffusion process in the reverse direction [40] for K timesteps z 0 → z K . Further, latent diffusion models employ classifier-free [59] guidance for text-conditioned generation, wherein the diffusion process is run twice, with text conditioning and unconditionally using a null-text token. To encourage accurate reconstruction of the original image, we follow Mokady et al. [20] to use the initial noisy diffusion trajectory (generated with a small guidance scale) as a pivot and update ∅ k , the null-text embedding at timestep k , to minimize a reconstruction mean square error between the predicted latent code ˆ z k and the pivot z k , using the default large guidance scale recommended for classifier-free guidance. This helps bring the backward diffusion trajectory close to the original image encoding z 0 , and thus achieve faithful reconstruction. Let S k -1 (ˆ z k , ∅ k , c ) denote one step of deterministic DDIM sampling [60]. We optimize:

Image editing: Checks and balances. We find that image editing using prompt-to-prompt with null-text inversion is highly sensitive to a specific hyperparameter which controls the fraction of diffusion steps for which self-attention maps for the original image are injected. Let f denote this fraction. The optimal value of f varies according to the degree of change, with larger changes (say editing the background or weather) requiring a small value. In service of making our algorithm fully automated, we follow prior work [43] to automatically tune this hyperparameter: we sweep over a range of values and threshold based on a CLIP [28] directional similarity metric [61], which measures the consistency in the change across images and captions in embedding space. Let E I and E T denote the CLIP image and text encoders. The CLIP directional similarity criterion ϕ ( . ) is given by:

Algorithm 1 Generating Language-guided Counterfactual Images

1: Input: Test set T ={(image x , label y ) } M 1 , model M , image captioner C , perturber LLM P , latent diffusion model L , maximum perturbations per image N , edit similarity threshold ϵ , image quality threshold τ

Finally, we ensure that the generated image is more similar to the edited rather than the original caption in CLIP's [28] embedding space. Algo. 1 details our full approach.

4 Experiments

In Section 4.1, we overview our experimental setup, describing the data, metrics, baselines, and implementation details used. Next, we present our results (Section 4.2), comparing the performance of a diverse set of pretrained models on the subset of the ImageNet test set, and on our generated counterfactual test sets. Additionally, we analyze the sensitivity of models to different types of edits and demonstrate the applicability of our method in deriving class-level insights into model bias.

4.1 Experimental Setup

Dataset. We evaluate LANCE on a subset of the ImageNet [2] validation set. Specifically, we study the 15 classes included in the Hard ImageNet benchmark [62]. Models trained on ImageNet have been shown to rely heavily on spurious features ( e.g. context) to make predictions for these classes, which makes it an ideal testbed for our approach. We consider the original ImageNet validation sets for these 15 classes, with 50 images/class, as our base set. Further, testing our approach on ImageNet has a few other advantages: i) The dataset contains naturally occurring spurious correlations rather than manually generated ones ( e.g. by introducing a dataset imbalance). ii) Testing our method on ImageNet images rather than constrained settings used in prior work e.g. celebrity faces in CelebA allows us to validate its effectiveness in more practical settings. iii) Finally, using ImageNet allows us to stress-test a wide range of pretrained models.

Metrics. We report the model's accuracy@k ( k ∈ { 1 , 5 } ) over the original test set T and generated counterfactual test set T ′ . For model M , we are interested in understanding the drop in accuracy@k over the counterfactual test set, compared to the original test set. We define this metric as follows:

However, it is possible for an intervention to alter model confidence without leading to a change in its final prediction. As a more fine-grained measure, we also report the absolute difference in predicted model confidence for the ground truth class over the original and counterfactual images. For a single instance, we define this as: ∆ p ( y GT | x ) = | p ( y GT | x ) -p ( y GT | x ′ ) |

Finally, to evaluate realism we also report the FID [63] score of our generated image test sets and perplexity of the generated and perturbed captions under LLAMA-7B [53].

Subject

Object

Domain

Background

Adjective

Perturbation type

0

10

20

30

40

50

%

p

(

yGT

|

x

) (

)

resnet50

vit_b_16

clip

convnext_base

Implementation Details. We use BLIP-2 (for image captioning), LLAMA-7B [53] (for structured caption perturation) with LoRA finetuning [54], StableDiffusion [16] (for text-to-image generation), and pretrained models from TIMM [64]. We use PyTorch [65] for all experiments. For finetuning LLaMA-7B, we concatenate the query, key, and value matrices per layer and approximate the updates to the resulting weight matrix using a rank-8 decomposition. We finetune the model for 37.5k steps updating the weights every 32 steps. For more details see appendix.

Baselines. In the absence of prior empirical work for our experimental setting, we compare the performance of LANCE to the following:

Original (Reference): The original, unmodified ImageNet test set, as a point of reference.

Reconstructed (Control): The reconstruction of the original ImageNet test set, as a control set

iii) LANCE-R (Baseline, Ours): As a baseline, we design a simple random caption perturbation strategy that randomly masks out a word in the original caption and replaces it with a different word using a masked language model [66]. To ensure meaningful perturbations, we impose two additional constraints: the word being modified should not be a stop word (to minimize wasteful edits e.g. changing 'a' to 'the'), and that the CLIP [28] similarity between the generated image and the new word should exceed its similarity to the original word.

4.2 Results

Across pretrained models, LANCE generates more challenging counterfactuals than baselines.

Our goal is to discover a set of example images that are especially challenging for a given model. Therefore, Table 2 reports top-1 and top-5 performance of multiple fixed models across two baseline test sets and our own (LANCE). We find that every model studied has significantly lower performance on our proposed generated test set. We compare this both to the standard ImageNet test set as well as a baseline variant of our method that uses random caption perturbations (LANCE-R). Finally, we verify that performance loss is not due to the image generation process by demonstrating that performance on a reconstructed test set is nearly identical to the original test set.

We provide example LANCE perturbations and generated images in Figure 5. You can find an example image for each perturbation type. In some cases, such as the domain edit (changing from

a photo of a howler monkey sitting on top of a tree trunk next to a green forest filled with trees

a sculpture of a howler monkey sitting on top of a tree trunk next to a green forest filled with trees

Predictions

capuchin (13.85%)

spider monkey (11.29%)

Predictions

chimpanzee (76.83%)

gorilla (8.05%)

siamang (5.32%)

a photo of a Couple of howler monkey sitting on top of a tree next to each other in a forest

a

photo

of a

Couple of howler

monkey sitting on top

of a tree next to each

other in a

bamboo forest

Predictions

capuchin (13.85%)

spider monkey (11.29%)

titi (3.94%)

Predictions

chimpanzee (76.83%)

gorilla (8.05%)

siamang (5.32%)

beaver (0.67%)

a photo of a group of husky dogs pulling a dog sled across a snow covered field with people in the background

a photo of a group of schnauzer pulling a dog sled across a snow covered field with people in the background

Predictions:

dogsled (53.04%)

Eskimo dog (27.14%)

Siberian husky (10.17%)

malamute (9.37%)

elkhound (0.05%)

Predictions:

standard schnauzer (80.38%) giant schnauzer (27.14%) mini schnauzer (6.14%) Tibetan terrier (0.45%) English sheepdog (0.26%)

a photo of a young child

in a swimming suit and

a

silver swimming cap

leans over the edge of a

swimming pool

a photo of a young child

in a swimming suit and

a

yellow swimming cap

leans over the edge of a

swimming pool

Predictions:

bathing cap (94.71%)

swimming trunks (2.02%)

tub (0.82%)

maillot (0.46%)

maillot (0.21%)

Predictions:

bathing cap (36.00%)

tub (18.81%)

killer whale (7.23%)

swimming trunks (5.81%)

great white shark (5.47%)

Edit subject

Edit object

a photo of a young

boy hanging from a

metal bar on a

snowy

playground

a photo of a young boy hanging from a metal bar on a playground

Predictions:

ski (23.24%)

pole (15.26%)

horizontal bar (14.13%)

parallel bars (13.67%)

shovel (9.95%)

Predictions:

horizontal bar (68.15%)

parallel bars (31.84%)

swing (0.01%)

balance beam (0.00%)

pole (0.00%)

Edit background

Edit domain

Edit adjective

titi (3.94%)

beaver (0.67%)

a photo to a sculpture) significant predictive changes result - from correctly identifying a howler monkey to incorrectly predicting a chimpanzee. As another example, the subject edit (changing the dogs from husky to schnauzer) prohibits the model from accurately predicting dogsled, conceivably because few schnauzers appeared pulling a sled in the training data.

Perturbation sensitivity varies by perturbation type and model. Now that we have confirmed that our generated test set provides challenging examples to assess our models, we may further analyze the impact of different types of perturbations. Figure 3 reports the relative change in predictive performance in response to different perturbation types. We find that changes to the background have the highest impact, followed by domain. Further, some models are more sensitive to LANCE images, with ResNet-50 [1] having the most sensitivity.

LANCE can be used to derive class-level insights into model bias. To do so, we compute the L1 distance between CLIP features for the original and edited words, and run K-Means clustering. We then visualize the clusters with the highest ∆ p ( y GT | x ) , corresponding to high sensitivity to a given change. Figure 6 illustrates examples. As seen, our method can also be used to surface class-level failure modes to inform downstream mitigation strategies. Importantly, we stress that while some failure modes are corroborated by other diagnostic datasets [62], ( e.g. using color-based context-cues to predict 'howler monkey'), others go beyond the capabilities of conventional IID test set-based diagnosis ( e.g. models relying on dog breed to predict 'dog sled'; see Figs. 1, 11).

Are generated images and perturbed captions realistic? As we are generating a new test set, it is important to verify that any performance changes are not due to an impact on the realism of our images. In Table 4 we report the FID scores of our generated images and the perplexity scores of our generated captions and observe both to be within a reasonable range. Further, we reiterate that using our pipeline to reconstruct the original image results in a test set where each model measures near identical performance (Table 2).

a photo of a young

boy sitting in the

back seat of

a

car

with a teddy

bear strapped to the

seat belt

a photo of a young

boy sitting in the

back seat of

a

bus

with a teddy

bear strapped to the

seat belt

Predictions:

seat belt (97.22%)

car mirror (0.97%)

Predictions:

seat belt (52.49%)

forklift (5.93%)

lumbermill (3.94%)

sliding door (3.50%)

chain saw (1.73%)

a photo of a woman

wearing a seat belt

sitting in the back

seat of a

car

Predictions:

seat belt (96.06%)

car mirror (0.58%)

CD player (0.48%)

dial telephone (0.23%)

hair spray (0.12%)

Predictions:

seat belt (19.59%)

hair spray (6.73%)

crossword puzzle (3.40%)

cardigan (3.24%)

wig (2.90%)

a photo of a woman

wearing a seat belt

sitting in the back

seat of a

bus

a

painting

of a woman

in a plaid jacket and

sunglasses making a

hush sign with her

finger on her lips

a

photo

of a woman in

a plaid jacket and

sunglasses making a

hush sign with her

finger on her lips

Predictions:

sunglasses (2.89%)

sunglass (1.89%)

mask (0.73%)

Predictions:

sunglasses (15.44%)

sunglass (10.81%)

pajama (1.55%)

bow tie (0.99%)

a

photo

of a man

wearing sunglasses and

a hat with a reflection

of himself in the

glasses of his face

a

painting

of a man

wearing sunglasses and

a hat with a reflection

of himself in the

glasses of his face

Predictions:

sunglass (42.41%)

sunglasses (41.44%)

ski mask (3.40%)

seat belt (2.36%)

snorkel (1.76%)

Predictions:

ski mask (20.37%)

mask (15.47%)

snorkel (7.78%)

sunglasses (3.93%)

sunglass (3.11%)

Predictions:

sunglass (40.52%)

sunglasses (28.76%)

sweatshirt (11.54%)

wig (1.12%)

trench coat (0.85%)

a

photo

of a woman in

a red coat and

sunglasses with her

arms on her sides

Predictions:

sweatshirt (47.77%)

wig (6.01%)

sunglass (3.80%)

cloak (3.49%)

comic book (2.66%)

a

painting

of a woman

in a red coat and

sunglasses with her

arms on her sides

a photo of a young boy

sitting in the back

seat of a

car wearing

sunglasses, a seat

belt, and a hoodie

a photo of a young boy

sitting in the back

seat of a

bus wearing

sunglasses, a seat

belt, and a hoodie

Predictions:

seat belt (56.94%)

sunglass (19.35%)

sunglasses (14.83%)

sleeping bag (2.23%)

limousine (0.99%)

Predictions:

sunglass (36.10%)

sunglasses (29.95%)

seat belt (19.74%)

sleeping bag (3.47%)

stretcher (1.38%)

Class: sunglasses, Intervention: photo

→

painting,

𝐆𝐓

x) = 𝟏𝟖%

Class: seat belt, Intervention: c

𝐚𝐫 →

bus,

𝐆𝐓

x) = 𝟑𝟏%

Class: howler monkey, Intervention: change color, 𝐀𝐯𝐠. 𝚫𝐩 𝐲 𝐆𝐓 x) = 𝟒𝟎%

Predictions:

spider monkey (0.26%)

capuchin (0.10%)

indri (0.01%)

Predictions:

spider monkey (73.30%)

langur (1.13%)

colobus (0.69%)

capuchin (0.62%)

a photo of howler a

monkey sitting on a

tree branch in a tree,

with

leaves

and

branches in the

background

a photo of howler a

monkey sitting on a

tree branch in a tree,

with

orange

leaves and

branches in the

background

Predictions:

A.black bear (46.32%)

wild boar (15.17%)

sloth bear (12.43%)

Predictions:

howler monkey (24.19%)

siamang (19.67%)

spider monkey (13.10%)

colobus (6.14%)

titi (5.92%)

a photo of a howler

monkey walking across

a lush

green field

next to a bushy area

with trees

a photo of a howler

monkey walking across

a lush

pink field

next to a bushy area

with trees

Predictions:

chimpanzee (0.59%)

gorilla (0.51%)

titi (0.21%)

orangutan (0.17%)

Predictions:

chimpanzee (3.14%)

gorilla (1.48%)

titi (0.22%)

a photo of a black

monkey sitting on top

of a tree filled

with

green

leaves

next to a tree

a photo of a black

monkey sitting on top

of a tree filled

with

snow-covered

leaves next to a tree

LANCE's checks & balances are highly effective. Next, we evaluate the accuracy of the caption perturber at making the type of edit specified by the prompt ( e.g. correctly changing only the domain for a domain edit). To do so, we manually validate a random subset of 500 captions and their perturbations (100 per type) generated by the LANCE. We observe an overall accuracy of 89%, with the object (84%) and adjective (82%) types achieving the lowest accuracies: we find that most failures for these types result from incorrect identification of the object or adjective in the sentence. We note that even in failure cases ( e.g. changing an irrelevant word), the edits made are still reasonable.

Next, we evaluate the efficacy of our caption editing checks and balances. We first label the same subset of 500 examples by hand with the correct action (filter / no filter) and compare against LANCE. Across perturbation types, LANCE achieves both high precision and recall, with a low overall false positive (2.8%) and false negative (1.6%) rate. Of these, a majority of false negatives (edits that alter the ground truth that we fail to catch) are for the subject and object types: we find these typically

change the ground truth inadvertently e.g. for a 'keyboard space bar' image of a typewriter, the edit typewriter → painting erroneously passes our filter (paintings do not usually have space bars).

Human evaluators validate the realism and efficacy of LANCE-generated edits. We conduct a human evaluation along 5 axes: i) Image Realism (1-5, 5=best): How easy is it to tell that the counterfactual image was generated by AI? ii) Edit success (1-5, 5=best): Does the generated image correctly incorporate the edit? iii) Image fidelity (1-5, 5=best): Are all the changes made relevant to accomplishing the edit? iv) Label consistency (Yes/No): Is the original image label still plausible for the generated image?, and v) Ethical issues (Text input): Is the generated image objectionable, or raise ethical concerns around consent, privacy, stereotypes, demographics, etc.?

We collect responses from 15 external respondents for a random subset of 50 <image, ground truth, generated caption, perturbed caption, counterfactual image> tuples (10 per-perturbation type), and report mean and variance. We include a screenshot of our study interface in Fig. 13, and report results in Table 4. As seen, images generated by LANCE are rated to be high on realism (4.2/5 on average) and fidelity (4.5/5 on average), and slightly lower on edit quality (3.7/5 on average). Of these, we find that background edits score lowest on edit success, due to sometimes altering the wrong region of the image. Further, generated images have high label consistency (95%), verifying the efficacy of LANCE's checks and balances. Finally, ∼ 2% of images are found to raise ethical concerns, which we further analyze below.

Ethical Issues. On inspection of the images marked as objectionable in our human study, we find that most arise from bias encoded in the generative model: e.g. for the edit woman → lifeguard, the generative model also alters the perceived gender and makes the individual muscular. Further, for people → athletes, the model also alters the person's perceived race. The model also sometimes imbues stereotypical definitions of subjective characteristics such as 'stylish' ( e.g. by adding makeup). To address this, we manually inspect the entire dataset of 781 images for similar issues and exclude 8 additional images. Going forward, we hope to ameliorate this by excluding subjective text edits and using text-to-image models with improved fairness [69] and steerability [57].

Failure modes and limitations. In Fig 12, we visualize LANCE's 5 most frequent failure modes: a) Label-incompatible text edit: e.g. black howler monkey → purple howler monkey (howler monkeys are almost always black or red). b) Poor quality image edit: Images with low realism, which are largely eliminated via our checks and balances. c) Image edit that inadvertently changes the ground truth: e.g. typewriter → stapler for the class 'space bar' (staplers do not contain space bars). d) Inherited bias: e.g. changing perceived gender based on stereotypes. e) Meaningless caption edit: e.g. boy in back-seat driver in the back seat. Such failures can confound an observed performance drop but this is largely mitigated by their low overall incidence.

LANCE also has a few intrinsic limitations. Firstly, it leverages several large-scale pretrained models, each of which may possess its own inherent biases and failure modes, which is highly challenging to control for. Secondly, intervening on language may exclude more abstract perturbations that are not easily expressible in words. However, we envision that LANCE will directly benefit from the rapid ongoing progress in the generative capabilities [70, 53] and steerability [57] of image and text foundation models, paving the road towards robust stress-testing in high-stakes AI applications.

Acknowledgements. This work was supported in part by funding from NSF #2144194, Cisco, Google, and DARPA LwLL. We thank Simar Kareer for help with figures, members of the Hoffman Lab for project feedback, and the volunteers who participated in our human study.

References

K. He, X. Zhang, S. Ren, and J. Sun, 'Deep residual learning for image recognition,' in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 770-778, 2016.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. , 'Imagenet large scale visual recognition challenge,' International journal of computer vision , vol. 115, pp. 211-252, 2015.

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, 'The cityscapes dataset for semantic urban scene understanding,' in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 3213-3223, 2016.

T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, 'Microsoft coco: Common objects in context,' in Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pp. 740-755, Springer, 2014.

D. Hoiem, S. K. Divvala, and J. H. Hays, 'Pascal voc 2008 challenge,' World Literature Today , vol. 24, 2009.

Z. Liu, Z. Miao, X. Zhan, J. Wang, B. Gong, and S. X. Yu, 'Large-scale long-tailed recognition in an open world,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 2537-2546, 2019.

I. Misra, A. Gupta, and M. Hebert, 'From red wine to red tomato: Composition with context,' in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 1792-1801, 2017.

C. Sakaridis, D. Dai, and L. Van Gool, 'Semantic foggy scene understanding with synthetic data,' International Journal of Computer Vision , vol. 126, pp. 973-992, 2018.

X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, 'Moment matching for multi-source domain adaptation,' in Proceedings of the IEEE/CVF international conference on computer vision , pp. 1406-1415, 2019.

D. Hendrycks and T. Dietterich, 'Benchmarking neural network robustness to common corruptions and perturbations,' arXiv preprint arXiv:1903.12261 , 2019.

B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, 'Do imagenet classifiers generalize to imagenet?,' in International conference on machine learning , pp. 5389-5400, PMLR, 2019.

P. Kattakinda, A. Levine, and S. Feizi, 'Invariant learning via diffusion dreamed distribution shifts,' arXiv preprint arXiv:2211.10370 , 2022.

P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, et al. , 'Wilds: A benchmark of in-the-wild distribution shifts,' in International Conference on Machine Learning , pp. 5637-5664, PMLR, 2021.

J. Pearl, Causality . Cambridge university press, 2009.

A. Ramesh, M. Pavlov, G. Goh, S. Gray, C. Voss, A. Radford, M. Chen, and I. Sutskever, 'Zero-shot text-to-image generation,' in International Conference on Machine Learning , pp. 8821-8831, PMLR, 2021.

R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, 'High-resolution image synthesis with latent diffusion models,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 10684-10695, 2022.

B. Kawar, S. Zada, O. Lang, O. Tov, H. Chang, T. Dekel, I. Mosseri, and M. Irani, 'Imagic: Text-based real image editing with diffusion models,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. , 'Language models are few-shot learners,' Advances in neural information processing systems , vol. 33, pp. 1877-1901, 2020.

A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or, 'Prompt-to-prompt image editing with cross attention control,' in International Conference on Learning Representations , 2022.

R. Mokady, A. Hertz, K. Aberman, Y. Pritch, and D. Cohen-Or, 'Null-text inversion for editing real images using guided diffusion models,' arXiv preprint arXiv:2211.09794 , 2022.

J. Li, D. Li, S. Savarese, and S. Hoi, 'Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,' arXiv preprint arXiv:2301.12597 , 2023.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. , 'Training language models to follow instructions with human feedback,' Advances in Neural Information Processing Systems , vol. 35, pp. 27730-27744, 2022.

E. Wong, S. Santurkar, and A. Madry, 'Leveraging sparse linear layers for debuggable deep networks,' in Proceedings of the 38th International Conference on Machine Learning (M. Meila and T. Zhang, eds.), vol. 139 of Proceedings of Machine Learning Research , pp. 11205-11216, PMLR, 18-24 Jul 2021.

S. Singla and S. Feizi, 'Salient imagenet: How to discover spurious features in deep learning?,' in International Conference on Learning Representations , 2022.

Z. Li and C. Xu, 'Discover the unknown biased attribute of an image classifier,' in The IEEE International Conference on Computer Vision (ICCV) , 2021.

S. Singla, B. Nushi, S. Shah, E. Kamar, and E. Horvitz, 'Understanding failures of deep networks via robust feature extraction,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 12853-12862, 2021.

S. Singla, A. M. Chegini, M. Moayeri, and S. Feiz, 'Data-centric debugging: mitigating model failures via targeted data collection,' arXiv preprint arXiv:2211.09859 , 2022.

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. , 'Learning transferable visual models from natural language supervision,' in International conference on machine learning , pp. 8748-8763, PMLR, 2021.

S. Eyuboglu, M. Varma, K. K. Saab, J.-B. Delbrouck, C. Lee-Messer, J. Dunnmon, J. Zou, and C. Re, 'Domino: Discovering systematic errors with cross-modal embeddings,' in International Conference on Learning Representations , 2022.

S. Jain, H. Lawrence, A. Moitra, and A. Madry, 'Distilling model failures as directions in latent space,' in International Conference on Learning Representations , 2023.

Y. Zhang, J. Z. HaoChen, S.-C. Huang, K.-C. Wang, J. Zou, and S. Yeung, 'Diagnosing and rectifying vision models using language,' in The Eleventh International Conference on Learning Representations , 2023.

R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, 'Shortcut learning in deep neural networks,' Nature Machine Intelligence , vol. 2, no. 11, pp. 665-673, 2020.

L. A. Gatys, A. S. Ecker, and M. Bethge, 'A neural algorithm of artistic style,' arXiv preprint arXiv:1508.06576 , 2015.

L. A. Gatys, A. S. Ecker, and M. Bethge, 'Image style transfer using convolutional neural networks,' in Proceedings of the IEEE conference on computer vision and pattern recognition , pp. 2414-2423, 2016.

T. Karras, S. Laine, and T. Aila, 'A style-based generator architecture for generative adversarial networks,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 4401-4410, 2019.

R. Abdal, Y. Qin, and P. Wonka, 'Image2stylegan: How to embed images into the stylegan latent space?,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 4432-4441, 2019.

R. Abdal, Y. Qin, and P. Wonka, 'Image2stylegan++: How to edit the embedded images?,' in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 8296-8305, 2020.

K. Crowson, S. Biderman, D. Kornis, D. Stander, E. Hallahan, L. Castricato, and E. Raff, 'Vqgan-clip: Open domain image generation and editing with natural language guidance,' in Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXVII , pp. 88-105, Springer, 2022.

A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mcgrew, I. Sutskever, and M. Chen, 'Glide: Towards photorealistic image generation and editing with text-guided diffusion models,' in International Conference on Machine Learning , pp. 16784-16804, PMLR, 2022.

P. Dhariwal and A. Nichol, 'Diffusion models beat gans on image synthesis,' Advances in Neural Information Processing Systems , vol. 34, pp. 8780-8794, 2021.

A. Hertz, K. Aberman, and D. Cohen-Or, 'Delta denoising score,' arXiv preprint arXiv:2304.07090 , 2023.

T. Brooks, A. Holynski, and A. A. Efros, 'Instructpix2pix: Learning to follow image editing instructions,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

A. Haque, M. Tancik, A. A. Efros, A. Holynski, and A. Kanazawa, 'Instruct-nerf2nerf: Editing 3d scenes with instructions,' arXiv preprint arXiv:2303.12789 , 2023.

A. Sauer and A. Geiger, 'Counterfactual generative networks,' in International Conference on Learning Representations .

G. Jeanneret, L. Simon, and F. Jurie, 'Diffusion models for counterfactual explanations,' in Proceedings of the Asian Conference on Computer Vision , pp. 858-876, 2022.

G. Jeanneret, L. Simon, and F. Jurie, 'Adversarial counterfactual visual explanations,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

M. Zemni, M. Chen, É. Zablocki, H. Ben-Younes, P. Pérez, and M. Cord, 'Octet: Object-aware counterfactual explanations,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

J. Luo, Z. Wang, C. H. Wu, D. Huang, and F. De la Torre, 'Zero-shot model diagnosis,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

X. Li, Y. Chen, Y. Zhu, S. Wang, R. Zhang, and H. Xue, 'Imagenet-e: Benchmarking neural network robustness via attribute editing,' in Proceedings of the IEEE conference on computer vision and pattern recognition , 2023.

O. Wiles, I. Albuquerque, and S. Gowal, 'Discovering bugs in vision models using off-the-shelf image generation and captioning,' in NeurIPS ML Safety Workshop , 2022.

J. Vendrow, S. Jain, L. Engstrom, and A. Madry, 'Dataset interfaces: Diagnosing model failures using controllable counterfactual generation,' arXiv preprint arXiv:2302.07865 , 2023.

L. Dunlap, A. Umino, H. Zhang, J. Yang, J. E. Gonzalez, and T. Darrell, 'Diversify your vision datasets with automatic diffusion-based augmentation,' arXiv preprint arXiv:2305.16289 , 2023.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. , 'Llama: Open and efficient foundation language models,' arXiv preprint arXiv:2302.13971 , 2023.

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, 'Lora: Low-rank adaptation of large language models,' arXiv preprint arXiv:2106.09685 , 2021.

N. Reimers and I. Gurevych, 'Sentence-bert: Sentence embeddings using siamese bert-networks,' arXiv preprint arXiv:1908.10084 , 2019.

B. Wallace, A. Gokul, S. Ermon, and N. Naik, 'End-to-end diffusion latent optimization improves classifier guidance,' in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pp. 7280-7290, October 2023.

L. Zhang, A. Rao, and M. Agrawala, 'Adding conditional control to text-to-image diffusion models,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 3836-3847, 2023.

C. H. Wu and F. De la Torre, 'A latent space of stochastic diffusion models for zero-shot image editing and guidance,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 7378-7387, 2023.

J. Ho and T. Salimans, 'Classifier-free diffusion guidance,' arXiv preprint arXiv:2207.12598 , 2022.

J. Song, C. Meng, and S. Ermon, 'Denoising diffusion implicit models,' arXiv preprint arXiv:2010.02502 , 2020.

R. Gal, O. Patashnik, H. Maron, G. Chechik, and D. Cohen-Or, 'Stylegan-nada: Clip-guided domain adaptation of image generators,' arXiv preprint arXiv:2108.00946 , 2021.

M. Moayeri, S. Singla, and S. Feizi, 'Hard imagenet: Segmentations for objects with strong spurious cues,' in Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2022.

M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, 'Gans trained by a two time-scale update rule converge to a local nash equilibrium,' Advances in neural information processing systems , vol. 30, 2017.

R. Wightman, 'Pytorch image models.' https://github.com/rwightman/pytorch-image-models , 2019.

A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. , 'Pytorch: An imperative style, high-performance deep learning library,' Advances in neural information processing systems , vol. 32, 2019.

V. Sanh, L. Debut, J. Chaumond, and T. Wolf, 'Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter,' ArXiv , vol. abs/1910.01108, 2019.

A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. , 'An image is worth 16x16 words: Transformers for image recognition at scale,' arXiv preprint arXiv:2010.11929 , 2020.

Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, 'A convnet for the 2020s,' in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 11976-11986, 2022.

C. Zhang, X. Chen, S. Chai, C. H. Wu, D. Lagun, T. Beeler, and F. De la Torre, 'Iti-gen: Inclusive text-to-image generation,' in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 3969-3980, 2023.

D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach, 'Sdxl: improving latent diffusion models for high-resolution image synthesis,' arXiv preprint arXiv:2307.01952 , 2023.

Appendix

A Dataset Details

Assets and license. All source images belong to the ImageNet dataset [2], which is distributed under a BSD-3 license that permits research and commercial use.

Statistics. We now provide an overview of the counterfactual test set generated via our method. After adding the checks and balances described in the main paper to ensure caption and image quality after editing, we approximately double the size of the HardImagenet dataset (from 750 to 1531). We provide a per-class breakdown of the generated samples in Figure 7: we find that our method is able to generate a larger number of counterfactuals for certain categories ( e.g. 'swimming cap') than others ( e.g. 'hockey puck'), presumably because one of two possible reasons: i) the structured caption perturber is able to generate several more plausible variations for certain classes, or ii) the checks and balances in our image editing pipeline filters out many more examples of certain classes due to not being of sufficiently high quality. We note that while we create this dataset as a proof-of-concept for our method, scaling it up to include additional classes and concepts is very straightforward, and simply requires finetuning the perturber on (a small number of) additional caption editing examples, which are relatively easy to collect.

B Per-class Analysis

In the main paper, we quantified performance in aggregate and per perturbation-type. We now investigate per-class performance over generated counterfactuals in Figure 8. We find that the counterfactuals generated for certain categories ( e.g. 'ski' or 'swimming cap') are significantly more challenging across models than others.

Do class-level insights generalize to the full dataset? Recall that in Figure 3 of the main paper we derived class-level insights into model bias, for example finding that the accuracy of ResNet-50 models at recognizing 'sunglasses' drops signficantly if the data domain is changed from 'photo' to 'painting'. A natural question then is whether such insights generalize directly to other classes. In Figure 9, we apply this intervention to the entire HardImageNet dataset and report the per-class drop in top-1 accuracy when going from the original test set to the generated counterfactual test set. As seen, across all but one category ('baseball player'), performance drops, often significantly ( e.g. 'balance beam' and 'hockey puck').

dogsled

seat belt

ski

volleyball

hockey puck

keyboard space bar

howler monkey

balance beam

patio

miniskirt

sunglasses

swimming cap

snorkel

gymnastic horizontal bar

baseball player

0

20

40

60

80

100

120

Count

35

50

58

60

22

14

54

57

45

50

59

125

45

92

15

miniskirt

seat belt

dogsled

hockey puck

volleyball

ski

keyboard space bar

patio

howler monkey

swimming cap

balance beam

baseball player

sunglasses

snorkel

gymnastic horizontal bar

Class

0

20

40

60

80

100

LANCE top-1 acc.

ResNet-50

ViT-B/16

CLIP

ConvNext

C Additional Qualitative Results

We provide additional qualitative examples generated via LANCE in Figure 10. We demonstrate 3 examples for each perturbation type: subject, object, background, domain, and adjective. Some of these examples are particularly interesting e.g. that the model's confidence of there being a 'balance beam' present goes down significantly if a 'coach' is pictured doing a handstand (top row, middle ). Similarly, the probability of the image containing a 'patio' under the model goes down considerably if benches are replaced by bicycles (row 2, left ). Similarly, the model can no longer recognize a 'dogsled' if the weather conditions become misty (row 3, left ). Finally, the model is much less confident in recognizing a 'seatbelt' if the color of its buckle changes from silver to gold (bottom row, middle ). We note that these are only image-level insights and require further investigation across images of a given class to draw conclusions of systematic model bias.

D List of Prompts

In Table 5 we provide an exhaustive list of the prompts we use for both GPT-3.5 turbo [18] and for finetuning LLAMA [53]. As seen, despite their simplicity our prompts are able to elicit the desired behavior from both models. We find that including a couple of examples in the prompt greatly improves performance.

Sunglasses

Dog Sled

Ski

Swimming Cap

Balance Beam

Gymnastic Horizontal Bar

Patio

Miniskirt

Volleyball

Baseball Player

Snorkel

0

5

10

15

20

Drop in top-1 accuracy (%)

11.77

5.23

8.85

12.50

8.48

5.67

9.08

20.32

3.51

2.18

22.22

4.29 4.86

-2.94

2.39

E Analyzing Failure Modes

Despite the considerable number of checks and balances that we insert into our pipeline, while running it end-to-end automatically and at scale, we find that a few poor counterfactuals still fall through the cracks. We provide qualitative examples of representative failures modes in Figure 12, that we group into five categories:

Inconsistent with GT. LANCE sometimes generates edits that, while being reasonable in isolation, generates an image that is inconsistent with the ground truth label. For example, 'howler monkeys' are always black or blonde in color, and so a purple howler monkey is statistically unlikely.

ii) Meaningless caption edit. In some cases, the structured perturber makes an edit that is visually meaningless, e.g. changing photo to video , while our pipeline is constrained to image generation.

iii) Poor quality image edit. Though we find this to occur very rarely due to our image quality filters, a few poor quality image edits are still retained e.g. accidentally cropping the limb of a person.

iv) Changing GT. A few edits inadvertently change the ground truth. For example, for the category 'space bar', changing a typewriter to a stapler also ends up removing the space bar.

Image generation bias. Finally, we find that the latent diffusion model's inherent bias also sometimes seeps through, e.g. when changing an image of a woman to a middle-aged woman (ground truth category: 'miniskirt'), the diffusion model also subtly changes the woman's attire to presumably be closer to attires commonly worn by middle-aged women in its training data, rather than changing physical age markers alone: in this case, the image cannot strictly be considered a counterfactual as multiple attributes are being intervened on.

F Implementation Details

We run all experiments on a single NVIDIA A40 GPU. All the models that we evaluate are trained with supervised learning on ImageNet-1K [2]. We include additional implementation details for hyperparameters used by LANCE for caption and image editing in Table 6.

a photo of a

car

is

parked on the porch of

a house with a covered

porch and a car parked

in front of it

Predictions:

patio (57.82%)

mobile home (18.06%)

picket fence (9.84%)

sliding door (7.36%)

rocking chair (1.21%)

Predictions:

mobile home (29.76%)

patio (16.50%)

picket fence (10.33%)

gas pump (8.23%)

bicycle-for-two(4.94%)

a photo of a

bicycle

is

parked on the porch of

a house with a covered

porch and a car parked

in front of it

Predictions:

crutch (82.80%)

ski (16.11%)

pole (0.32%)

snorkel (0.11%)

parallel bars (0.08%)

a

painting

of a

woman standing in

the snow with skis

on her feet and a

hat on her head

a

photo

of a woman

standing in the snow

with skis on her

feet and a hat on

her head

Predictions:

ski (99.64%)

pole (0.09%)

crutch (0.08%)

shovel (0.04%)

alp (0.03%)

a photo of a couple

of

men standing next to

each other on a beach

with snorkels on their

heads

a photo of a couple

of

women standing

next to each other on

a beach with snorkels

on their heads

Predictions:

snorkel (99.55%)

sea snake (0.19%)

scuba diver (0.14%)

sea cucumber (0.05%)

stingray (0.02%)

Predictions:

snorkel (99.95%)

sea snake (0.02%)

scuba diver (0.02%)

sea cucumber (0.00%)

a photo of a man

holding a baseball

bat on top of a lush

green

soccer

field

a photo of a man

holding a baseball bat

on top of a lush

green

field

Predictions:

ballplayer (96.95%)

baseball (2.18%)

football helmet (0.32%)

plunger (0.17%)

Predictions:

soccer ball (92.90%)

ballplayer (6.17%)

baseball (0.58%)

football helmet (0.12%)

golf ball (0.10%)

Predictions:

balance beam (91.91%)

parallel bars (4.42%)

horizontal bar (3.63%)

volleyball (0.02%)

basketball (0.01%)

a photo of

a

woman

doing a

handstand on a balance

beam in a gym with

people watching

Predictions:

volleyball (94.92%)

horizontal bar (3.36%)

parallel bars (0.76%)

balance beam (0.38%)

basketball (0.26%)

a photo of

a

coach

doing a

handstand on a balance

beam in a gym with

people watching

a photo of a

pair of

sunglasses and a

stuffed animal

a drawing of a

pair of

sunglasses and a

stuffed animal

Predictions:

Pencil box (25.39%)

Purse (7.92%)

Stethescope (7.92%)

Hair slide (6.73%)

Sunglasses (4.85%)

Predictions:

Hair slide (9.92%)

Spindle (8.34%)

Pencil box (8.02%)

Thimble (6.01%)

Wooden spoon (5.81%)

a photo of a group of

chairs sitting on top

of a

cement floor next

to a palm tree

a photo of a group of

chairs sitting on top

of a

marble floor

next to a palm tree

Predictions:

dining table (29.20%)

limousine (27.44%)

patio (11.99%)

home theater (5.29%)

restaurant (3.43%)

Predictions:

dining table (41.14%)

tub (10.64%)

patio (9.99%)

washbasin (7.18%)

bathtub (3.62%)

a photo of a man in

sunglasses and

a

fur

selfie in the

mountains with snow

on the ground

a photo of a man in

sunglasses and

a

hoodie

taking a

selfie in the

mountains with snow on

the ground

Predictions:

sunglasses (52.59%)

sunglass (27.44%)

alp (3.11%)

sweatshirt (2.54%)

sunscreen (1.79%)

Predictions:

sunglasses (44.52%)

sunglass (24.31%)

alp (6.57%)

valley (1.83%)

ski (1.70%)

Edit subject

Edit object

Edit background

Edit domain

Edit adjective

a photo of a woman in a

red jacket is pulling

two dogs on a leash in

the

misty

woods

a photo of a woman in

a red jacket is

pulling two dogs on a

leash in the

woods

Predictions:

toy terrier (44.61%)

Eskimo dog (12.72%)

malamute (6.38%)

Siberian husky (3.96%)

French bulldog (2.77%)

Predictions:

Eskimo dog (31.83%)

dogsled (16.98%)

malamute (12.72%)

Siberian husky (5.94%)

a photo of a woman in

a

print

leotard

a photo of a woman

in a

black leotard

Predictions:

balance beam (61.49%)

horizontal bar (16.56%)

pole (9.16%)

parallel bars (8.77%)

maillot (1.01%)

Predictions:

pole (44.26%)

balance beam (21.45%)

horizontal bar (18.91%)

parallel bars (4.48%)

maillot (3.23%)

a

drawing

of a man

riding a dog sled

pulled by two dogs on

a snow covered field

with trees in the

background

a

photo

of a man

riding a dog sled

pulled by two dogs on

a snow covered field

with trees in the

background

Predictions:

dogsled (99.32%)

Eskimo dog (0.46%)

Siberian husky (0.17%)

malamute (0.03%)

timber wolf (0.00%)

Predictions:

dogsled (99.11%)

Eskimo dog (0.31%)

Siberian husky (0.19%)

malamute (0.12%)

horse cart (0.08%)

a photo of

a

gold

seat belt and

a cell phone

a photo of a

seat

belt

and a cell phone

Predictions:

seat belt (97.08%)

buckle (0.82%)

digital watch (0.50%)

scabbard (0.39%)

cellular telephone (0.30%)

Predictions:

buckle (46.82%)

digital watch (22.26%)

seat belt (5.80%)

lighter (5.25%)

ballpoint (1.84%)

a photo of a man with

a

beard wearing

sunglasses and an

orange shirt and a hat

a photo of a man with

a

moustache

wearing

sunglasses and an

orange shirt and a hat

Predictions:

sunglass (41.61%)

binoculars (16.89%)

sunglasses (11.79%)

web site (7.41%)

bubble (1.57%)

Predictions:

sunglass (55.14%)

sunglasses (23.37%)

binoculars (11.75%)

web site (2.42%)

Predictions:

ballplayer (75.83%)

baseball (24.17%)

rifle (0.00%)

racket (0.00%)

football helmet (0.00%)

a photo of a

young

boy

in a

baseball uniform

holding a bat on top

of a baseball field

Predictions:

ballplayer (72.80%)

baseball (27.18%)

rifle (0.00%)

a photo of a

young

girl

in a

baseball uniform

holding a bat on top

of a baseball field

Predictions:

puck (88.25%)

ski (1.44%)

scoreboard (1.43%)

go-kart (0.72%)

unicycle (0.41%)

Predictions:

puck (98.36%)

scoreboard (0.92%)

broom (0.16%)

ballplayer (0.07%)

digital clock (0.06%)

a photo of a group of

young men playing a

game of ice hockey on

an

outdoor

ice rink

a photo of a group of

young  men  playing  a

game of ice hockey on

an

indoor

ice

rink

6

8

BOHE

旧

28

20

Class: Dog Sled, Intervention: change dog breed to Malamute / Samoyed (both sled-dog breeds)

a photo of a group

of

dogs

pulling a dog

sled across a snow-

covered ground next to

a person

a photo of a group

of

samoyed

dogs pulling

a dog sled across a

snow-covered ground

next to a person

Predictions:

dogsled (80.58%)

Eskimo dog (14.41%)

Siberian husky (4.56%)

malamute (0.42%)

Predictions:

Samoyed (84.13%)

malamute (7.60%)

Eskimo dog (4.88%)

Siberian husky (1.89%)

dogsled (1.25%)

a photo of a group of

people standing around

a

dog

pulling a dog

sled with a dog on it

a photo of a group of

people standing around

a

samoyed

dog pulling a

dog sled with a dog on it

Predictions:

dogsled (80.58%)

Eskimo dog (14.41%)

Siberian husky (4.56%)

malamute (0.42%)

Predictions:

dogsled (36.84%)

Eskimo dog (26.01%)

Samoyed (21.58%)

Siberian husky (2.89%)

ski (1.99%)

a photo of a group

of

huskies pulling a dog

sled across a snowy

field with people in

the background

a photo of a group

of

malamutes pulling a

dog sled across a snowy

field with people in

the background

Predictions:

dogsled (52.89%)

Eskimo dog (26.85%)

malamute (10.08%)

Siberian husky (9.86%)

Predictions:

malamute (85.67%)

dogsled (6.07%)

Eskimo dog (4.90%)

Siberian husky (3.33%)

a photo of a group

of

dogs

pulling a dog

sled across a snow-

covered ground next to

a person

a photo of a group

of

malamute dogs

pulling a dog sled

across a snow-covered

ground next to a person

Predictions:

dogsled (80.58%)

Eskimo dog (14.41%)

Siberian husky (4.56%)

malamute (0.42%)

Predictions:

dogsled (40.12%)

Eskimo dog (38.65%)

Siberian husky (18.99%)

malamute (2.22%)

Inconsistent with GT

Poor quality image edit

Changing GT

Image generation bias

A photo of a woman

standing in front of

a door with her hands

on her hips and a

purse in her hand

A photo of a middle-

aged woman standing in

front of a door with

her hands on her hips

and a purse in her

hand

Predictions:

miniskirt (99.22%)

cowboy boots (0.38%)

trench coat (0.10%)

wig (0.05%)

sandal (0.04%)

Predictions:

doormat (34.79%)

Cardigan (13.87%)

miniskirt (9.42%)

suit (5.62%)

Meaningless caption edit

a photo of

a

teenager

doing

a trick on a

beam on a blue

surface with a

blue background

a photo of

a

woman

doing a

trick on a beam

on a blue

surface with a

blue background

Predictions:

balance beam (98.26%)

horizontal bar (1.30%)

parallel bars (0.22%)

maillot (0.06%)

maillot (0.05%)

Predictions:

balance beam (50.78%)

pole (37.46%)

horizontal bar (8.01%)

parallel bars (1.42%)

bow (0.28%)

a photo of

a

black

monkey

sitting on top of a

tree filled with

green leaves next to

a tree

a photo of

a

purple

monkey

sitting on top of a

tree filled with

green leaves next to

a tree

Predictions:

chimpanzee (0.59%)

gorilla (0.51%)

titi (0.21%)

orangutan (0.17%)

Predictions:

macaque (22.84%)

langur (14.55%)

capuchin (6.62%)

chimpanzee (6.34%)

a photo of

a

typewriter

sitting

on top of a stack of

books on top of a

wooden table

a photo of

a

stapler

sitting on

top of a stack of

books on top of a

wooden table

Predictions:

typewriter keyboard (52.63%)

space bar (29.02%)

computer keyboard (11.14%)

notebook (1.35%)

Predictions:

modem (16.56%)

notebook (7.92%)

printer (6.95%)

hard disc (5.12%)

a

video

of two

little girls hanging

from a metal bar in

a playground area

with buildings in

the background

a

photo

of two

little girls hanging

from a metal bar in

a playground area

with buildings in

the background

Predictions:

horizontal bar (99.93%)

parallel bars (0.07%)

swing (0.00%)

pole (0.00%)

balance beam (0.00%)

Predictions:

horizontal bar (99.85%)

parallel bars (0.14%)

swing (0.01%)

pole (0.00%)

balance beam (0.00%)

UDAT

[Instructions: Please read first!]

LANCE is a tool to uncover vulnerabilities in a trained image classifier by generating challenging test cases.

It does so by editing a single visual aspect in a given image (described in text), while leaving the rest of it unchanged.

The goal of this study is to validate the quality of (a random subset of 50) images generated via LANCE, along five axes

1) Realism: How

realistic

is the image generated by LANCE? (or, how easy it to tell that this image was created by an AI model?)

2) Edit success: How well is the intended edit

reflected

in the generated image? (or, how well does the new caption match the generated image?)

3) Image fidelity: How similar is the generated image to the starting image in

all other aspects

? (or, is the generated image sufficiently similar to the original image?)

4) Label consistency: Is the original image label still

plausible

for the generated image?  (or, is the ground truth label (Col F) still a reasonable match to the generated image?)

5) Ethical considerations: Is the generated image

objectionable

, or raise ethical concerns around consent, privacy, stereotypes, demographics, in any way

Original Image

(real)

Image Caption

(generated)

LANCE

(generated)

LANCE Caption

(generated + edited)

Ground

Truth

Image

Realism

(1-5)

Edit

success

(1-5)

Image

Fidelity

(1-5)

Label

Consistency

(check if yes)

Ethical

Issues

(check if yes)

Details of ethical Issues

(e.g. NSFW, negative stereotypes,

discriminatory etc.)

a photo of a boy

sitting in the back

seat of a car

wearing a seat belt

a photo of a teenage

boy sitting in the

back seat of a car

wearing a seat belt

seat belt

4

5

5

a photo of a man

sitting in the back

seat of a car

wearing a seat belt

a photo of a middle-

aged man sitting in

the back seat of a

car wearing a seat

belt

seat belt

value

min. caption length

20

max. caption length

100

repetition penalty

1.0

ϵ

0.5

decoding strategy

beam search

beam size

5

clustering strategy

K-Means

number of clusters

5

(a)

Caption Generation and Clustering

value

learning rate

3e-4

effective batch size

128

grad. accumulation steps

32

weight decay

0.0

1024

lora rank [54]

8

α

[54]

16

0.05

lora weights [54]

qkv

(b)

LLAMA Finetuning

value

Stable Diffusion [16] version

1.4

image resolution

512x512

LR scheduler

DDIM scheduler [60]

[60]

0.00085

beta_end

[60]

0.012

beta_schedule

[60]

scaled linear

diffusion steps [19]

50

attention_replace_edit

[19]

2

cross_replace_steps

[19]

0.8

self_replace_steps

[19]

[0

.

4

,

0

.

5

,

0

.

6

,

0

.

7

,

0

.

8

,

0

.

9]

guidance scale

7.5

τ

[42]

0.2

img sim. threshold [42]

0.7

img-text sim. threshold [42]

0.2

(c)

Image Editing

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents a method for stress-testing visual classifiers. The key idea is to use language to generate counterfactual images. Their approach can be summarized into four steps: 1) take an image input; 2) get its caption from a captioning model (BLIP-2); 3) perturb the caption by changing some words using ChatGPT and fine-tuned LLAMA; 4) generate new images based on perturbed captions using text inversion of Stable Diffusion. On the ImageNet dataset, they showed that various models such as ResNet-50 and ViT-B consistently achieve much lower performance on their generated new ImageNet compared to the original ImageNet, and the generated ImageNet enables surfacing model biases.\n\nSTRENGTHS:\n1. This paper targets a novel direction --- stress-test visual models, which is important in real-world vision applications.\n\n2. The proposed method is technical sound, and the generated images from their pipeline look good and realistic.\n\n3. The paper conducted experiments on the realistic ImageNet dataset instead of simple synthetic datasets, showing the practical values of their method in realistic scenarios.\n\n4. This paper is generally well-written and easy to understand.\n\nWEAKNESSES:\n1. There is no baseline to compare, and the numbers are hard to justify their method's effectiveness. The main quantitive results from this paper showed that various vision models achieve much lower performance on their generated ImageNet dataset. However, it is unclear whether the generated images are 100% correct. If the text perturbation happens to change the class or introduce new classes to the image, the model prediction is expected to change and the performance will drop. Therefore, a human study showing the label validity of generated images is particularly necessary.\n\n2. There are some existing works proposing very similar directions. For example, [1] is almost the same as this work, which uses textual reversion to convert an image into a token and compose this token with natural language and generate new images to test vision models. [2] uses off-the-shelf image captioning and generation models to discover model bugs. While [1] can be viewed as concurrent work, it would be helpful if the authors could discuss the similarity and differences between this work and [1-2].\n\n3. From Figure 6 last row, changing the caption from \"car\" to \"bus\" generates almost the same images, but model predictions change a lot. The change may be because generated images have specific patterns that are not perceptible to humans, which cannot reflect the actual errors using real images. \n\n4. The stress test will be limited by language itself, as language is hard to represent certain properties of objects, such as orientation and lighting. However, this is also pointed out by the authors and is minor considering the advantage of using language.\n\n[1] Dataset interfaces: Diagnosing model failures using controllable counterfactual generation. https://arxiv.org/pdf/2302.07865.\n\n[2] Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioning. https://arxiv.org/pdf/2208.08831.\n\nQUESTIONS:\n- Algorithm 1: \n  - Line 7 duplicate definition of \"c\" (previously used as the class, now used as the caption)\n  - Line 13: \">\" should be \"<\"\n\n- Equation 2: \"|\" in the denominator should be \"||\"\n\n- Line 282: what is the percentage for the generated counterfactuals to be semantically inconsistent with the ground truth label?\n\nLIMITATIONS:\nSee weaknesses. I'm happy to increase my rating if these are well addressed, especially 1 and 2.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-06"}

comments: SUMMARY:
The paper proposes a way to generate new data to evaluated the robustness of model. It first captions an image, then changes part of it, then generates an image from the modified caption. The model is then evaluated on these new images to see how robust they are to things like background changes.

STRENGTHS:
The idea is interesting and is an easy way to generate new evaluation data. The paper is well written and easy to follow. The approach would be a valuable tool to evaluate models.

WEAKNESSES:
Since it is an automated benchmark, it is unclear how good the quality is. It relies on a lot of parts working right (captioning model, caption changing model and image generation model).  It would be good to have some human evaluation in Table 2 to see how well humans do on this set.

QUESTIONS:
How can the quality of the generated dataset be evaluated? How do we know it is generating a good benchmark?

LIMITATIONS:
Yes.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-06

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a new testing protocol LANCE for existing vision models.\nIt adopts LLM-based caption perturbation and DDPM based image editor to generate counterfactual samples. \nSpecifically, for each image, LANCE first use a pre-trained captioner (BLIP-2) to generate the corresponding caption. \nThen it adopts an LLM to perturb the caption with different factors (subject, object, background, etc.).\nThe goal of perturbation is to generate a caption corresponding to a counterfactual image with semantics preserved.\nWith the perturbed caption, the author generates another image with counterfactual aspects and similar looking with the original image.\nThe authors observed a significant performance drop when testing image classification models on these generated images.\nFurthermore, the authors also present a detailed analysis based on the perturbation factors.\n\nSTRENGTHS:\n1. The authors establish an interesting pipeline to generate counterfactual hard examples for testing vision models. It involve an LLM-based image caption perturbator and a ddpm-based image editor. Qualitative results show that this pipeline can generate some impressive counterfactual images. \n2. The authors present a detailed analysis on the performance degradation when testing on the generated counterfactual images.\n\nWEAKNESSES:\n1. The authors should present more quantitative statistics about the counterfactual examples generation pipeline, for example:\n    1. What is the success rate of an LLM to generate a plausible perturbed caption (GPT-3.5 & Finetuned LLAMA-7B). Also, I noticed that you are using Nvidia A40 (48GB), thus a comparison between different LLAMA variants (+13B, e.g.) are also welcomed. \n    2. What is the success rate of DDPM-based image editing? I'm specially interested in that since I observe poor quality when using StableDiffusion to generate common images (compared to other advanced AIGC models).\n2. Are all newly generated images (781 images, according to the supp) verified manually by human? If so, how many worker-hours are required to generate these samples?\n\nQUESTIONS:\nSee Weakness.\n\nLIMITATIONS:\nSee Weakness.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-14"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to use recent advances in text-to-image generation/editing to design evaluation by generating \"counterfactual\" data. The main idea of the method is to make semantically irrelevant (to the output class) edits to an image and observe its effects on the model predictions. A variety of models including CNNs, and ViT (trained classically as well as via language supervision) are tested, which show that such editing operation can indeed make a \"hard\" test set where most tested methods struggle to do well.\n\nSTRENGTHS:\n[S1] Great Presentation Clarity: The paper is very well written and the details about what has gone into the final data-generation scheme are presented in sufficient detail. The experimental setup, the algorithm, and the implementation of the baseline models are clearly explained and should be easily reproducible independently.\n\n\n[S2] Good evaluation/results: The proposed model, based on the experiments presented, show a huge potential for using counterfactual generation of images to stress-test models. While the paper only makes a handful of edit types, there is a possibility to design hyper-specific probes (does change clothing color change models predictions about a person's profession?) that can be designed for specific applications.\n\nWEAKNESSES:\n[W1] Need more discussion of scope and their limitations: While the main idea is sound and the descriptions are clear, the evaluation/discussion leaves a lot to be desired. Most of the evaluation is done \"end-to-end\" which does not give much insight into the success of the various stages involved in the process. Some example concerns that I have:\n\n- The success of checks and balances (L155 and L185) is not verified/discussed. Do they always work? When do they fail? What are the effects of their failing?\n- What is the diagnostic prowess of this method? Besides showing that the edit-based perturbations does indeed make models flip their predictions (which has also been previously shown via adverserial noise injection), what insights can we draw from this type of stress testing? Does the results of this type of probe correlate with other tests (perhaps for bias/fairness and/or other diagnostic datasets? Do the edits represent (in the paper's view), the realm of possible naturally occurring variations in background/unrelated content in the picture? If not, can this still diagnose/predict the model's robustness in the real world? Or, is this just another form of discovering \"adversarial attack\" on the model predictions?\n\nThere is only a very short limitation section at the end of the conclusion section. I would have liked to see a more open discussion about exactly what (in view of the paper) can this method tell us about a model and what it cannot. \n\n\n[W2] No manual/external/independent validation: The quality of the edits, and the accuracy of the labels post-edit are not validated by any external/manual evaluation. We only see decreased overall accuracies for popular methods. While a simple baseline of random edit is tested, it is not clear whether a noise perturbation/other simpler perturbation techniques could also result in similar loss. Regardless of the loss in accuracy produced, it is also important to know whether the editing process leaves any unwanted artifacts. E.g., In the headline image, there is no actual sled visible, and the prediction of it likely comes from context. As such, it is unsurprising that changing the dog breed to pomeranian (an unlikely breed to be pulling sleds, also both \"husky\" and \"pomeranian\" are actual categories by themselves) would result in flipped predictions. To be clear, I am not claiming that this is \"OKAY\" behavior, just that perhaps the mechanism is \"label noise\" in Imagenet rather than \"context bias\" (Since there is nothing but the context that can help us make predictions). I am claiming, however, that this is a different category of error than other examples in the paper. Both are important, and the proposed method is likely to help uncover both, but at a lack of external validation about the accuracy of the labels before and after the edit operation. Some questions that I'd have liked validated externally:\n- Can the accuracy of the \"perturber\" be measured independently?\n- What is the accuracy of the ground truth labels post-edit? Are they truly unchanged? Are the methods presented in L185 and L155 (re. checks and balances) enough? Do they work? Have they avoided this issue successfully?\n- Since there is a pervasive error in Imagenet (https://arxiv.org/abs/2103.14749), does this method have any correlation with the noisy labels? In other words, is this intervention making \"easy to predict\" examples harder or primarily just the \"confusing\" ones?\n\nQUESTIONS:\nPlease see the weaknesses above\n\nLIMITATIONS:\nPlease see the weaknesses above", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-19"}