Paper: BackBench: Are Vision Language Models Resilient to Object-to-Background Context? (ICLR-2024)

Reviewer comment (Novelty):
The generation pipeline is built on existing methods. The whole method is not entirely new. It seems that the only changes being made are appending some pre-defined descriptions of the background. Figure 3 clearly presents more diverse backgrounds than other examples in the supplementary results. I believe there’s more to explore on how to generate diverse backgrounds other than randomizing seeds.

Author response:
**On using Existing Foundational Models.** Given the large-scale multi-modal training of recent vision-language models and their generalization capabilities, there is a need for evaluation benchmarks going beyond simple common corruptions [1] or adversarial perturbations [2] to better understand their behavior in the wild. The main motivation of our proposed work is to study how changes in the object background can affect the performance of various uni/multi-modal models. To this end, the proposed framework utilizes existing foundational models such as, SAM [3] and BLIP-2 [4] for visual and textual conditioning. Recent relevant works also utilize large-scale pretrained models but for global perturbations, while ours specifically studies the background changes impact on models’ performance. For instance, LANCE [5] (NeurIPS’23) employs the use of pretrained Large Language Models alongside an already proposed image editing technique [6] to induce global modifications in the real images. Such works underscore a shared trend in the field: the creative re-purposing of existing foundational models to forge new ways to evaluate the robustness of uni/multi-modal models. Different from LANCE, our contribution lies in how we have utilized the complementary strengths of different foundational models for our proposed object-to-background benchmarking. Our work preserves the semantics, shape, angle, and color of the original object while changing its composition to the background in the real images.  By combining the capabilities of current image-to-text, image-to-segment and text-to-image-models, we have created a framework that is significantly more adept at handling complex background changes. Furthermore, we use our framework to generate adversarial background changes by optimizing the visual and textual latents of the diffusion model. Our rigorous set of experiments on vision tasks such as, image classification, object detection and segmentation are designed to validate the effectiveness of our framework across uni/multi-modal models.

**Generating Diverse Backgrounds.** Kindly note that in Figure 3 (main paper), we generate diverse, realistic background changes by a variety of prompts using ChatGPT rather than random seeds. We provide these ChatGPT prompts in Table 7 (revised Appendix A.3). This demonstrates the effectiveness of our text prompt-based diversity in image generation. Additionally, we provide further qualitative results of this capability in the appendix, specifically in Figures 8 and 9 (see revised Appendix A.3), where we display a range of results achieved using these varied text prompts. 

We show the effect of random seeds in Figure 7 in Appendix A.3, where we use BLIP-2 captions for the original image as a text prompt. We fixed this text prompt and generated images with different random seeds. We observe from rows 2, 3 and 5 of Figure 7, that altering the seed while keeping the same caption does not yield significant background diversity across the generated samples. Furthermore, using random seeds only does not provide the fine-grain control needed to benchmark vision-based models, which is our main objective. These observations highlight the importance of our designed textual prompts to generate diverse object-to-background changes for benchmarking current vision-based models.

Reviewer comment (Experiments):
The title discusses the robustness of vision language models while most experiments focus on classification and detection, and there’s only one real multi-modal task, image captioning.

Author response:
**On Studying Robustness of Vision-Language Models.**  In this work, our focus has been on examining vision language models through various lenses, including classification, detection, segmentation, and image captioning. Specifically, we have conducted evaluations of vision-language models like CLIP and EVA-CLIP for zero-shot classification tasks (Table 2 in the main paper, Table 10 in the revised Appendix). For the case of segmentation, our attention has been on assessing the performance of the multi-modal segmentation model SAM (Table 8 in the revised Appendix A.6). Moreover, for the task of image captioning, we have evaluated the robustness of the image-to-text BLIP-2 model (Table 4 in the main paper). Along with these, we study different uni-modal CNNs and vision transformers trained naturally and adversarially. As suggested, we will modify the title to avoid any confusion such as "Backbench:  Are Vision Models Resilient to Object-to-Background Context?"

Reviewer comment (Experiments):
Latest transformer/CNN models should be considered, such as DeiT and ConvNeXt.

Author response:
**Evaluating Recent Vision Models.** As recommended, we have conducted evaluations on recent models such as DeiT[7] and ConvNeXt[8] in the Table below (also added in Section A.13 in the revised Appendix). We observe a consistent trend of decrease in performance to background changes across these models as well. We observe that DeiT-based models which distill the knowledge from a strong CNN-based model during training show overall improved performance across our object-to-background changes compared to ViT-based models (please see Table 1 in main paper).


|                      |        | DeiT   |        |         |            | CNN        |            |         |
|:--------------------|:------:|:------:|:------:|:-------:|:----------:|:----------:|:----------:|:-------:|
| Background (IN-Nat)  | DeiT-T | DeiT-S | DeiT-B | **Average** | ConvNeXt-T | ConvNeXt-B | ConvNeXt-L | **Average** |
| Original             | 96.36  | 99.27  | 99.41  | 98.34   | 99.07      | 99.21      | 99.40      | 99.22   |
| Class Label          | 94.18  | 96.85  | 97.74  | 96.25   | 97.60      | 97.51      | 97.51      | 97.54   |
| BLIP-2 Caption       | 89.33  | 94.29  | 95.07  | 92.89   | 94.64      | 94.82      | 95.47      | 94.97   |
| Color                | 80.96  | 89.48  | 91.11  | 87.13   | 92.11      | 93.58      | 93.58      | 93.09   |
| Texture              | 74.15  | 84.01  | 86.75  | 81.63   | 88.50      | 89.50      | 91.13      | 89.71   |
| Background (IN-Adv)  |        |        |        |         |            |            |            |         |
| Original             | 95.44  | 99.10  | 99.10  | 97.88   | 99.00      | 99.00      | 92.92      | 96.97   |
| Adversarial          | 20.40  | 29.62  | 34.81  | 28.27   | 32.88      | 42.52      | 48.60      | 41.33   |

*Comparison of recent CNN and transformer-based models (Top-1%) against our proposed object-background changes our IN-Nat and IN-Adv datasets. The table is also added in revised Appendix A.13 (Table 14).*

Furthermore, we also provide results on DINOv2-based models [9]. Our preliminary experiments with the DINOv2 models, as presented in Table below (see A.14 in revised Appendix), show that DINOv2 models which utilize registers (learnable tokens) [10] during training for learning more interpretable attention maps, provide more robustness to background changes, with significant improvement seen in the adversarial background changes.


|                      |       | DINOv2 |       |         |       | DINOv2 (with register) |       |         |
|:---------------------|:-----:|:------:|:-----:|:-------:|:-----:|:---------:|:-----:|:-------:|
| Background (IN-Nat)  | ViT-S | ViT-B  | ViT-L | **Average** | ViT-S | ViT-B                  | ViT-L | **Average** |
| Original             | 96.78 | 97.18  | 98.58 | 97.51   | 97.71 | 98.05                  | 99.14 | 98.30   |
| Class Label          | 94.62 | 96.02  | 97.18 | 95.94   | 95.55 | 96.44                  | 97.94 | 96.64   |
| BLIP-2 Caption       | 89.33 | 94.29  | 95.07 | 92.89   | 94.64 | 94.82                  | 95.47 | 94.97   |
| Color                | 83.85 | 89.68  | 93.31 | 88.94   | 85.88 | 91.15                  | 94.64 | 90.55   |
| Texture              | 83.63 | 89.08  | 92.44 | 88.38   | 84.98 | 91.03                  | 93.97 | 89.99   |
| Background (IN-Adv)  |       |        |       |         |       |                        |       |         |
| Original             | 95.12 | 96.50  | 98.10 | 96.57   | 97.91 | 97.80                  | 99.00 | 98.23   |
| Adversarial          | 54.31 | 71.62  | 80.87 | 68.93   | 58.30 | 76.21                  | 84.50 | 73.00   |


*Evaluation of Dinov2-based models on IN-Nat and IN-Adv dataset.  DINOv2 model with registers shows higher robustness to background changes, particularly in the presence of color, texture, and adversarial backgrounds. The table is also added in revised Appendix A.14 (Table 15).*

Reviewer comment (Evaluation):
There are limited analyses of the experimental results. A lot of results are presented but what are the conclusions?

Author response:
**Results and Conclusions.** Our results section provides a comprehensive analysis revealing that vision-based models are vulnerable to background variations. We observe:

1. Vison-based models are vulnerable to diverse background changes, such as texture and color, with the **most prone to adversarial background changes** (Table 1 in the paper, Tables 10,11 and 14 in the Appendix).
2. **Increasing the capacity of the model** for both CNN and transformer-based models helps in improving the robustness against varying background contexts. This indicates, that distilling from a more robust model can help in improving the robustness of small models (Table 1 in the main paper, Tables 10, 11, 14 and 15 in the revised Appendix).  Evidence of this is seen in the performance of DeiT-T, which, by distilling knowledge from a strong CNN-based model, shows improved robustness as compared to ViT-T (see Section A.13 in Appendix).
3. Our study indicates that **adversarially trained models have limited robustness.** While they perform well in scenarios with adversarial background changes, their effectiveness is limited for other types of object-to-background compositions, as seen in Figure 4 of our main paper. This highlights a significant gap in current model training approaches when it comes to dealing with diverse background changes. 
4. Object detection and segmentation models, which incorporate object-to-background context, display reasonably better robustness to background changes than classification models, as evidenced by quantitative and qualitative results (Table 5 and Figure 5 in the paper and Section A.5 in the Appendix).
5. Also, recent training approaches for vision transformer-based classification models that learn more interpretable attention maps [10] show improvement in robustness to background changes (Section A.14 in revised Appendix). This potentially indicates the importance of inducing object-to-background context during training as revealed by our work.
6. Furthermore, as shown in Table 2 (paper), models trained on large-scale datasets with more scalable and stable training show better robustness against background variations. 

In conclusion, our research highlights the need to evolve vision-based models that are capable of accurately discerning pertinent features in diverse settings, shifting away from reliance on background elements. This approach not only aligns with recent innovations in the field but also paves the way for enhancing the adaptability and robustness of vision-based models.

Reviewer comment (Novelty):
References to SAM and BLIP-2 should be added in Section 3.2.

Author response:
**Minor Changes**
We thank the reviewer and have incorporated references to SAM and BLIP-2 in Section 3.2 as recommended and provided a reference in Section 3.2 for a clearer understanding of the changes made to the textual prompt. In our approach, for adversarial background changes, we optimize both visual and textual latents of the diffusion model while for non-adversarial background changes, we employ the default inpainting diffusion pipeline.

Further, to improve clarity, we have revised Figure 2 in Section 3.1 of the main paper. Detailed insights and results mentioned above are now included in the appendix. Moreover, in the final revision, we plan to refine the results Section 4.1, aiming for greater clarity and ease of understanding.

Reviewer comment (Presentation):
The presentation of the paper can be improved. Some technical details are not very clear in Section 3.2 (see Question 1) and it’s unclear what changes have been made other than directly applying the existing models (see Question 2). Also, the presentation of Section 4 is a bit messy. A lot of experimental settings and results are presented while there is only one sub-section “4.1 Results”.

Author response:
**Minor Changes**
We thank the reviewer and have incorporated references to SAM and BLIP-2 in Section 3.2 as recommended and provided a reference in Section 3.2 for a clearer understanding of the changes made to the textual prompt. In our approach, for adversarial background changes, we optimize both visual and textual latents of the diffusion model while for non-adversarial background changes, we employ the default inpainting diffusion pipeline.

Further, to improve clarity, we have revised Figure 2 in Section 3.1 of the main paper. Detailed insights and results mentioned above are now included in the appendix. Moreover, in the final revision, we plan to refine the results Section 4.1, aiming for greater clarity and ease of understanding.

Reviewer comment (Experiments):
What “changes” are made from $\mathcal{T}$ to $\mathcal{T}’$? Just appending prompts about the background in Table 6?

Author response:
**Changes in Textual prompts.** Table 6 (Appendix) refers to the textual prompts used to generate background changes related to color, texture, and class name specifically. We also show the huge space of background changes that can be explored by showing qualitative samples in Figures 3 (main paper), 8, and 9 (Appendix), where we generate diverse realistic background changes using prompts varied generated using ChatGPT (see Table 7 in revised Appendix A.3).

Reviewer comment (Reproducibility):
In Section 3.2, are there any changes that have been made besides applying the inpainting diffusion model?

Author response:
**Minor Changes**
We thank the reviewer and have incorporated references to SAM and BLIP-2 in Section 3.2 as recommended and provided a reference in Section 3.2 for a clearer understanding of the changes made to the textual prompt. In our approach, for adversarial background changes, we optimize both visual and textual latents of the diffusion model while for non-adversarial background changes, we employ the default inpainting diffusion pipeline.

Further, to improve clarity, we have revised Figure 2 in Section 3.1 of the main paper. Detailed insights and results mentioned above are now included in the appendix. Moreover, in the final revision, we plan to refine the results Section 4.1, aiming for greater clarity and ease of understanding.

Reviewer comment (Evaluation):
My primary concern about the study, while already collaborative, is that there seems to be a lack of further analysis. For example, we do not know whether the image alteration process will create additional object information that will confuse the classification. Image classification models are only allowed to make one prediction for each image. Thus, any new visual objects created during the image manipulation could easily confuse them. To rule out this hypothesis, a causal analysis of the classification errors might be necessary. There could be other hypotheses that need to be ruled out, too.

Author response:
- **Possibility of extra objects in the Background:**  Kindly note that **a)** we use a pretrained diffusion model that is conditioned on a pretrained CLIP text encoder, this means that the generated output follows the latent space of the CLIP text encoder which is aligned with CLIP visual encoder. Therefore, we can measure the faithfulness of the generated sample w.r.t the textual prompt used to generate it. We can measure this by encoding the generated output and its corresponding text prompt within CLIP latent space. For a given sample, CLIP or EVA-CLIP performs zero-shot evaluation by measuring the similarity between embedding of class templates (e.g. 1000 templates of ImageNet class) with a given image. Thus, if we simply add the template for a textual prompt used to generate the object-to-background changes, then we can measure its alignment with the background changes. For instance, instead of using a “a photo of a {fish}” template for zero-shot classification, we add the **relevant template** that is with background change, such as “a photo of a {fish} in the vivid colorful background”.  In other words, the relevant template represents the object and background change we introduced. We validate this observation on the EVA-CLIP ViT-E/14+, a highly robust model. Using the class templates such as “a photo of a {}”, the model achieves 95.84% accuracy on the original images (IN-Nat dataset), which decreases to **88.33%** when our color background changes are applied. However, when using the **relevant template**,  the performance improves to **92.95%**, significantly reducing the gap between the performance on the original and color background changes from **7.51%** to **2.89%**. These results show that accuracy loss from background changes isn't due to unwanted background objects of other classes. Furthermore, we manually assess  2.89% of misclassified samples (very few samples, see Figures 28 and 29 in the revised Appendix A.10). These can be considered the hardest examples in our dataset. We observe that even in such hard cases the model's confusion often stemmed from the complex background patterns instead of the addition of unwanted objects. We observe a similar trend in the case of adversarial patterns as well (see Figure 30 in the revised Appendix A.10). **b)** Another empirical evidence of how our generated output closely follows the given textual prompts can be observed with BLIP-2 Caption of the original image. In this case, object-background change has similar results as compared to original images across different vision models (Table 2 in the main paper).

Reviewer comment (Evaluation):
The term resiliency to object-to-background seems solely measured by the accuracy drop of classification models on images altered with the proposed method. Is this metric well-calibrated? Will there be another metric that can quantify this effect? Does this metric correlate to the resiliency of models against other background change processes?

Author response:
The design choices discussed above, such as high-quality data filtering, strong visual guidance, and class-agnostic textual guidance, contribute to the well-calibrated results of our study. This indicates that our results using the conventional metrics such as classification accuracy are well calibrated as well in the context of our high quality of generated data as mentioned above. We note that these choices ensure that the models are primarily challenged by diverse changes in the background, rather than being misled by the presence of unwanted objects. This careful approach underlines the reliability of our findings and highlights the specific factors influencing model performance.

Reviewer comment (Novelty):
This article lacks innovation, and the proposed framework is a combination of existing methods, such as SAM, BLIP-2, and DDIM.

Author response:
**On Novelty.** The main motivation of our proposed work is to study how changes in the object background can affect the performance of various uni/multi-modal models. To this end, the proposed framework utilizes existing foundational models such as, SAM [1] and BLIP-2 [2] for visual and textual conditioning. Recent relevant works also utilize large-scale pretrained models but for global perturbations, while ours specifically studies the background changes impact on models’ performance. For instance, LANCE [3] (NeurIPS’23) employs the use of pretrained Large Language Models alongside an already proposed image editing technique [4] to induce global modifications in the real images. Such works underscore a shared trend in the field: the creative re-purposing of existing foundational models to forge new ways to evaluate the robustness of uni/multi-modal models. 

Different from LANCE, our contribution lies in how we have utilized the complementary strengths of different foundational models for our proposed object-to-background benchmarking. Our work preserves the semantics, shape, angle, and color of the original object while changing its composition to the background in the real images.  By combining the capabilities of current image-to-text, image-to-segment and text-to-image-models, we have created a framework that is significantly more adept at handling complex background changes. Furthermore, we use our framework to generate adversarial background changes by optimizing the visual and textual latents of the diffusion model. Our rigorous set of experiments on vision tasks such as, image classification, object detection and segmentation are designed to validate the effectiveness of our framework across uni/multi-modal models.

Reviewer comment (Experiments):
This article does not compare with the current SOTA method of adversarial and counterfactual manipulations.

Author response:
**Comparison with Recent State-of-the-art Method.**  As recommended, we compare with the recently proposed LANCE [3] (NeurIPS’23), which is relevant to our approach for benchmarking robustness. Their work leverages the capabilities of large language models to create textual prompts, facilitating diverse image alterations using the prompt-to-prompt method [4] and null-text inversion [5] for real image editing. We observe the following noticeable difference with LANCE as compared to our proposed approach:

1. LANCE relies on prompt-to-prompt method [4]  and null-text inversion [5] for real image editing which allows modification of only specific words in the textual prompt. This limitation can restrict the range of possible image transformations. In contrast, our method uses visual conditioning to better preserve the object semantics, allowing us the flexibility to use textual guidance to introduce diverse background changes.
2. LANCE makes global changes to a given input image which can alter object semantics, e.g., shape distortion (See Table 13 and Figures 25 and 26 in the revised Appendix A.9).
3. Furthermore, LANCE needs adjustments in attention-based manipulation hyperparameters, and the selection of optimal images based on CLIP scores for each sample which is not scalable to larger datasets. In contrast, our approach allows for the evaluation of vision-based models across a much broader range of vision tasks. On the task of classification, we provide an evaluation of 5.5k images across 582 classes of ImageNet (See Figure 32 in revised Appendix), as opposed to the 15 classes evaluated in LANCE.

We use open-sourced code from LANCE to compare it against our approach both quantitatively and qualitatively. We use a subset of 1000 images, named IN-ADV, for comparison. **a)** Quantitative comparison: We observe that our natural object-to-background changes including color and texture perform favorably against LANCE, while our adversarial object-to-background changes perform significantly better as shown in the table below.

| Method     | Object-to-background Change | ViT-T | ViT-S | Swin-T | Swin-S | Res-50 | Res-152 | DenseNet-161 | Average |
|:-----------|:----------------------------:|:-----:|:-----:|:------:|:------:|:------:|:-------:|:------------:|:-------:|
| Original   | --                          | 95.5  | 97.5  | 97.9   | 98.3   | 98.5   | 99.1    | 97.2         | 97.71   |
| LANCE [3]  | --                          | 80.0  | 83.8  | 87.6   | 87.7   | 86.1   | 87.4    | 85.1         | 85.38   |
| Ours       | Color                       | 67.1  | 83.8  | 85.8   | 86.1   | 88.2   | 91.7    | 80.9         | **83.37**   |
| Ours       | Texture                     | 64.7  | 80.4  | 84.1   | 85.8   | 85.2   | 90.1    | 80.3         | **81.55**   |
| Ours       | Adversarial                 | 18.4  | 32.1  | 25.0   | 31.7   | 2.0    | 28.0    | 14.4         | **21.65**   |

*Quantitative comparison (Top-1%) of our approach with LANCE. Our approach performs favorably well for benchmarking the robustness of ViTs and CNNs. The table is also added in the revised Appendix A.9 (Table 12).*


**b)** Qualitative comparison: Since LANCE relies on global-level image editing, thus tends to alter the object semantics and distort the original object shape in contrast to our approach which naturally preserves the original object and alters the object-to-background composition only. This can be observed in qualitative examples provided in Figures 25 and 26 in revised Appendix section A.9. We further validate this effect by masking the original images and LANCE-generated counterfactual images. We observe an accuracy drop from 97.71% to 84.35% across different models. Note that this is the first step of our approach, where the background is just masked but not optimized for any background changes. However, when the background is masked in LANCE-generated counterfactual images, overall accuracy drops from 97.71% to 71.57%. This shows that the LANCE framework has changed the original object during optimization which is also reflected in Figures 25 and 26 in revised Appendix Section A.9. Therefore, our proposed approach allows us to study the correlation of object to background changes without distorting the original object.

| Method    | Object-to-background Change | ViT-T | ViT-S | Swin-T | Swin-S | Res-50 | Res-152 | DenseNet-161 | Average |
|:----------|:----------------------------:|------:|------:|-------:|-------:|-------:|--------:|:-------------:|:--------:|
| Original  | --                          |  95.5 |  97.5 |   97.9 |   98.3 |   98.5 |    99.1 |         97.2 |   97.71 |
| Original  | Masked Background           |  70.5 |  86.1 |   84.2 |   87.6 |   87.2 |    91.2 |         83.7 |   84.35 |
| LANCE [3] | Masked Background           |  59.5 |  72.5 |   72.3 |   75.3 |   71.9 |    77.5 |         72.0 |   71.57 |

*Quantitative evaluation (Top-1%) of LANCE on IN-Adv shows its tendency to alter the original object in contrast to our approach which preserves the original object during optimization. The table is also added in revised Appendix A.9 (Table 13).*

Reviewer comment (Experiments):
An additional SAM model is used, which may not be a fair comparison with the SOTA methods.

Author response:
**On using SAM.** Kindly note that we use SAM [1] as our image-to-segment model for visual conditioning to *preserve the object semantics*. This aligns with our objective to study the robustness of vision-based models to object-to-background compositional changes. The requirement to preserve the object semantics needs an instance segmentation model like SAM, which if not used would result in editing the actual object (a limitation other global methods like LANCE suffer from).