Paper: CLIP Exhibits Improved Compositional Generalization Through Representation Disentanglement (ICLR-2024)

Reviewer comment (Reproducibility):
The human validation seems crucial in generating the proposed benchmark; however, there is no detailed description of how this was performed.

Author response:
**Point 2**: As outlined in Section 6.2 and visually depicted in Figure 3, our human evaluation protocol involved a straightforward yet effective method for ensuring the quality and relevance of the generated images. Two evaluators were assigned the task of assessing each image against two critical criteria to determine its suitability for inclusion in the dataset:

-**Object Recognition**: Any image where the central object was not clearly recognizable or was ambiguous was removed.

-**Attribute Visibility**: Images were excluded if the specified attributes were not evidently depicted.

Reviewer comment (Theory):
In section 1, the authors claim: "By assessing the captions in the training sets, we guarantee that none of the captions in our test dataset or similar captions are included in the CLIP training data." -- however, I couldn't find any empirical or theoretical evidence, nor existing reference for this claim.

Author response:
**Point 3**: Our methodology involved an initial generation of 30,000 attribute-object combinations. We then conducted a comprehensive search for these combinations within the captions of various datasets on which CLIP has been trained, such as LAION. This search was instrumental in identifying and eliminating any combinations that already existed within the training data. Even if the individual attribute and object words appeared at a distance in a caption, we eliminated the combination of them from our combinations.

Consequently, the remaining 12,000 combinations, which did not appear in the training data, were used to generate our test dataset. This rigorous process underpins our confidence in asserting that our test dataset comprises truly OOD examples relative to the CLIP training datasets.

Reviewer comment (Evaluation):
The human validation only asses for the plausibility of the noun-adjective composition, but are the images generated by DALLE following those compositions? Prior work has shown that Diffusion models "struggle to understand the composition of certain concepts, such as confusing the attributes of different objects or relations between objects"[1]. It is unclear if the generated dataset follows the attribute-noun composition, or falls into this category. See also [2].

Author response:
**Point 4**: We acknowledge the valid concerns raised about ensuring the integrity of the attribute-noun compositions in our generated images, and the potential issues with diffusion models identified in prior work. As detailed in Section 6.2 and shown in Figure 3, our human evaluation process was designed to directly address these concerns. The main motivation for human evaluation is that text-to-image models currently have limitations in generating coherent compositions. We first evaluated different models like Stable Diffusion variants, Deep Floyd, and DALL-E, and found DALL-E to be the most capable at generating compositions. However, since its accuracy is imperfect for every composition, we added human evaluation to our pipeline to remove low quality 
and ambiguous images, and also the images whose contents are not consistent with the specified attribute-object.

Reviewer comment (Novelty):
Most of the conclusions are prevalent in the literature (e.g., the diversity of training captions promotes compositionality [3]), and the mutual information analysis does not seem to provide additional insights [4, 5].

Author response:
**Point 5**:
While prior work has explored the relationship between training data diversity and compositional generalization, our study provides new analysis quantifying *how* diversity in the training data leads to improved disentanglement in model representations, thereby directly enabling better out-of-distribution (OoD) generalization.

Specifically, we conducted the first analysis (Section 3.3.1) that examines the impact of caption diversity and quality in training datasets on the mutual information (MI) between object and attribute tokens. We found that lower NMI indicates higher disentanglement between the two, which can improve the compositional generalization.

Building on this, we propose a way to measure the disentanglement between objects and attributes in the representation spaces of CLIP text and image encoders (Section 3.3.2). To our knowledge, this disentanglement has not been explored or validated before in CLIP models. We demonstrate models with more disentangled representations achieve higher performance on our new compositional OoD benchmark. We also provided evidence that the disentanglement is much stronger on the text representation, and provided some theoretical insights on how the disentanglement could be induced on the image representation from the text. 

Furthermore, we provide additional insights into CLIP disentanglement beyond our benchmark by evaluating on 3D Shapes datasets. Our work is the first to extensively examine and validate the connection between disentangled representations and compositional generalization performance of CLIP models.

Overall, our analysis and experiments on quantifying disentanglement and its impact on compositional generalization enhance the understanding of why and how CLIP models are able to generalize compositionally. We extend current knowledge by empirically demonstrating that diversity promotes compositionality through inducing disentangled representations.

Reviewer comment (Experiments):
Is there any particular reason why DINO-v2 and BEiT-v2 are mentioned briefly in the introduction, but no further analysis is done in the following sections?

Author response:
**Question**: Since DINO-v2 and BEiT-v2 are primarily image encoders without an accompanying text encoder, they fall outside the scope of our evaluation criteria. Most of our experimental design required models with text encoding capabilities to comprehensively assess and compare their performance.

Reviewer comment (Experiments):
"*the training dataset of the OpenAI CLIP has not been released, which makes designing a test set that has a truly different distribution from the training one challenging. We aim to address these issues in this paper, by focusing our attention on the compositional generalization in the single object setting, and designing an authentic test dataset to assess the training data characteristics and mechanisms in the models that lead to the OoD generalization.*" -- This contradicts the following statement where the authors claim that they verify that the ImageNet-AO images are not a part of several CLIP training dataset -- "*To ensure these combinations were not present in the CLIP training set, we conducted a thorough search and removed any combinations that were found.*"

Author response:
**Point 1**: It is true that the specific training dataset used by OpenAI for CLIP has not been publicly re-
leased. However, there are several multimodal datasets that have been made available and are commonly
used for training vision-language models, such as LAION, YFCC, CC, and DataComp. Our work utilizes
these publicly available datasets to construct a test as mentioned in Section 6.2. We also evaluate CLIP
models trained on these public training datasets.

Reviewer comment (Experiments):
"*By assessing the captions in the training sets, we guarantee that none of the captions in our test dataset or similar captions are included in the CLIP training data.*"  
Is this check done for all the other datasets considered in the paper as well (LAION, YFCC15m, CC12m, and DataComp)?

Author response:
**Point 2**: We have thoroughly checked all datasets referenced in our paper, including LAION, YFCC15m,
CC12m, and DataComp, to ensure that (attribute,object) pairs in our dataset are not present in the
captions of these CLIPs’ training data.
Since we want to evaluate models on novel compositions, we first found (attribute, object) pairs that
are not appear in any captions of the available training sets of CLIPs. For these novel (or extremely rare)
combinations, the images were also generated which makes the possibility of image overlap extremely
unlikely. For instance, we have combinations such as ‘hairy jeep,’ or ‘luminous golf cart,’ in our dataset,
as illustrated in Fig. 2, which are unlikely to be encountered on the web.
While we acknowledge the possibility that the model may have been exposed to similar images during
training, we can assert with confidence that the exact image and complete caption pairs used in our
test dataset are unique and represent new combinations that the model has not previously encountered.
This pairing approach ensures that, despite any potential familiarity with individual images, the model
is still being evaluated on its ability to generalize to new image-caption combinations, which we have
engineered to be out-of-distribution (OoD). Additionally, the principal experiments presented in our
paper, particularly in Section 3.1, as well as the supplementary experiments detailed in Sections 4.2, 6.3,
and 6.4, incorporate both image and text components within their methodologies. These experiments
do not solely concentrate on the image aspect.

Reviewer comment (Evaluation):
"*We also found that the CLIPs that show higher OoD generalization typically exhibit strong disentangled text representations. Furthermore, such CLIPs also enjoy a more disentangled image representation with respect to the attributes and objects as well.*" -- the experiments in the paper do hint at the above statement. But this does not necessarily imply the following: "*Specifically, a dataset with diverse compositions of attribute-objects facilitates a more disentangled text representation, which in turn induces a disentangled image representation through contrastive learning.*" It could be possible that diverse images lead to disentangled image representations as well.

Author response:
**Point 3**: The textual input is inherently disentangled, with adjective and object tokens being *distinctly separate*. In contrast, the visual inputs inherently exhibit entanglement; attributes and objects within images are naturally intertwined. By augmenting the diversity of textual data, we promote disentanglement in text representation, as the model learns to distinguish between the separate tokens for adjectives and objects due to their consistent and isolated presentation. However the entanglement in image representations persists because the attributes and objects are not presented as isolated features but as parts of an integrated whole.  Therefore, while increased diversity can enhance the model's exposure to varied visual concepts, it does not automatically lead to disentangled representations. These claims are also backed by Fig. 4, where we see that the text encoder usually exhibits a more pronounced disentanglement score compared with the image encoder. 

In the table below, we also compare a CLIP model with some image-only models that were trained on large-scale datasets or are among the top-performing models on ImageNet benchmarks. We evaluate these models on different disentanglement metrics  on the ImageNet-AO.

**Table: Disentanglement metrics on different Models**

| Model                     | z_diff ↑ | DCI-I ↑ | explicitness ↑ | DCIMIG ↑ |
|---------------------------|----------|---------|----------------|----------|
| CLIP-ViT-14-datacomp-xl   | 0.955    | 0.1834  | 0.9103         | 0.025    |
| DINOv2                    | 0.95     | 0.1787  | 0.9022         | 0.021    |
| Beit                      | 0.901    | 0.1463  | 0.884          | 0.016    |
| resnext101      | 0.948    | 0.1745  | 0.901          | 0.0209   |
| Effeicent-b7              | 0.899    | 0.1274  | 0.8531         | 0.0134   |

Reviewer comment (Evaluation):
"*To evaluate the degree of disentanglement in the training captions utilized by the CLIP, we conducted an analysis by measuring the normalized mutual information (NMI) between the object class and attribute tokens, whose domains are defined based on the captions in our generated dataset.*" -- Could the authors explain how the domains are defined based on the captions in the generated dataset? More details on how the NMI is measured would be helpful.

Author response:
**Point 4**:
 In the context of Normalized Mutual Information (NMI) in our experiments, 'X' represents the entire set of attributes that we utilized in generating our dataset, encompassing various descriptive elements such as texture. 'Y', on the other hand, corresponds to the set of objects used in generating our dataset, including a range of specific items or entities. These definitions of 'X' and 'Y' enable us to analyze the relationship between attributes and objects in our dataset as learned by the CLIP model.

$$
\\begin{aligned}
& \\mathrm{MI}(\\mathrm{X} ; \\mathrm{Y})=E_y[\\mathrm{H}(\\mathrm{X})-\\mathrm{H}(\\mathrm{X} \\mid \\mathrm{Y}=\\mathrm{y})] \\\\
& \\mathrm{H}(\\mathrm{X})=-\\sum_{i=1}^n p_i^* \\log \\left(p_i\\right) \\\\
& p\\left(x_i\\right)=\\frac{\\# \\text { captions that have } x_i}{\\sum_{j=1}^n \\text { \\#captions that have } x_j} \\\\
& p(x, y)=\\frac{\\# \\text { captions that have } x \\text { and } y}{\\sum_{x y} \\sum_y \\text { captions that have } x_i \\text { and } y_j} \\\\
& \\mathrm{H}(\\mathrm{X} \\mid \\mathrm{Y}=\\mathrm{y})=-\\sum_x p(x, y) * \\log \\left(\\frac{p(x, y)}{p(y)}\\right) \\\\
& \\mathrm{NMI}(\\mathrm{X} ; \\mathrm{Y})=\\frac{2 * M I(X ; Y)}{[H(X)+H(Y)]}
\\end{aligned}
$$

Reviewer comment (Evaluation):
Fig.4 - It is not clear how the disentanglement metrics are computed for the image encoder.

Author response:
**Point 5**: As mentioned in Section 3.3.2, we can consider two super latent factors corresponding to attributes and objects. For each vector of factors, we choose a image associated with those factors. We then calculate the embedding for that image. This gives us a set of representations and corresponding latent factors. Using these embeddings and factors, we can calculate different disentanglement metrics. To assess the randomness effects, we repeated this experiment multiple times,  and observed that the results show negligible variance. We also observed that if instead of random selection, we calculate embeddings of all images per relation and take the average, the results hold. We note that this is because the images associated with a certain relation have similar embeddings. For more details on how each metric is calculated, please refer to [2].

Reviewer comment (Experiments):
"*We aimed for a diverse set of class names to enhance the complexity of the generated images.*" -- It is not clear if all 1000 classes were used or only a subset. If a subset was used, how was this chosen?

Author response:
**Point 6**: As detailed in Section 6.2 of the Appendix, we utilized the entire spectrum of 1000 classes from
the ImageNet dataset.

Reviewer comment (Experiments):
"*This dataset was produced by creating compositional images via a text-to-image model, using an Attribute+Object template.*" -- Could the authors give more details/ a citation for the Attribute+Object template?

Author response:
**Point 7**:  We appreciate the opportunity to provide additional details on the methodology used to generate our dataset. As mentioned in Section 6.2 and illustrated in Figure 3, we employed a simple template structure, [ATTRIBUTE] [OBJECT], to create compositional images via a text-to-image model. This approach allowed us to systematically combine various attributes with objects to enrich the diversity and complexity of the images. Also this approach was used in previous works [3,4]. For concrete examples of these prompts, one may refer to Figure 2 in the main text, which showcases sample image captions generated using this template. Furthermore, a more extensive collection of prompts can be found in Figure 8 of the Appendix.

Reviewer comment (Experiments):
Could the authors provide details on where the 30 adjectives were chosen from?

Author response:
**Point 8**: Initially, we compiled an extensive list of adjectives using various sources, including linguistic websites and large language models like ChatGPT, to ensure a broad and inclusive initial selection. The primary list contained 50 attributes. Through iterative experimentation, we generated preliminary images for a range of objects paired with these adjectives. This practical evaluation was crucial as it allowed us to observe the representation of certain attributes within the images. We found that some adjectives, such as ‘fast’, did not visually translate well into the generated images. Consequently, these were omitted from the final list. The refinement process led us to a curated set of 30 adjectives that were visually discernible and thus suitable for our image generation objective. The full methodology for selecting attributes will be detailed in the Appendix.

Reviewer comment (Evaluation):
"*Lastly, human evaluation was used to validate the generated images, with images not closely aligning with their prompts removed. After this process, around 12000 combinations remained, for which we successfully generated near 50000 accurate, high-quality images.*" - The order of the two statements may have to be swapped? Could the authors provide details on how this human evaluation was done?

Author response:
**Point 9**: We have to clarify that by 'after this process ...,' we meant to report the number of generated images after the final step, which is the human inspection. Here we got total of 50k images from 12k combinations that passed the human evaluation criteria. The human inspection was conducted in two stages: First, before generating any images, we filtered the initial list of 30,000 combinations by searching for these combinations in the captions of training datasets and removing any matches, to ensure the remaining combinations were novel and feasible to depict visually. Second, after generating images for the remaining novel combinations using the text-to-image model, further human evaluation was necessary to eliminate unsatisfactory images where either the object was not clearly rendered or the intended attribute was not distinctly perceptible. Through this two-step refinement process, we narrowed down the dataset from 30,000 initial combinations to a final curated collection of around 12,000 novel  (attribute,object) combinations comprising 50,000 images that passed the human evaluation criteria.

Reviewer comment (Experiments):
In Fig.1, for a more fair comparison, the image-only models such as DINO-v2 and BEiT-v2 should also be trained on the datasets that were used for training CLIP (by using only the images, and ignoring the captions). Without matching at least the image datasets, there is not enough evidence to support the following statement - "*We interpret these findings as strong evidence that the inclusion of language supervision, particularly during CLIP training, positively impacts the model representation quality, hence making it possible to generalize to unseen compositions, despite the absence of such compositions in their training data.*"

Author response:
**Point 11**: As delineated in Section 3.2, the aim of our study was not to forge a direct and fair comparison between CLIP and image-only models like DINO-v2 and BEiT-v2. Instead, our focus was investigating the mechanism that causes the language supervision to improve  CLIP's out-of-distribution (OoD) generalization capabilities as mentioned in the title and abstract of our paper. We selected the supervised and self-supervised models for comparison based on their state-of-the-art ImageNet performance and their training on large-scale datasets, which allowed us to make a closer, albeit approximate, comparison given our practical constraints.
Only to see performance of other powerful foundation models (that are not based on language supervision), we show the results of these models on our proposed dataset too. 


The evidence supporting the role of language supervision in enhancing CLIP's performance is multifaceted, including:
- Variations in performance among models trained on Conceptual Captions (CC) and YFCC datasets, particularly with lower Normalized Mutual Information (NMI).
- A clear disentanglement at the representation level.
- A well-documented correlation between representation disentanglement and OoD generalization.

Our narrative underscores the distinctive benefits of language supervision and its impact on OoD accuracy, rather than on a direct performance comparison. Given the logistical challenges of compiling a dataset comparable to the one used to train CLIP, we opted to showcase the most relevant models to support our hypothesis.

Reviewer comment (Reproducibility):
Codes, models, and datasets must be open-sourced for the benefit of future research. Could the authors comment on this? Would these be released upon acceptance?

Author response:
At the end, about your question, we plan to open-source all codes and datasets upon the paper’s acceptance to
support future research.

Reviewer comment (Evaluation):
In 3.2, the statement 'We interpret these findings as strong evidence that the inclusion of language supervision, particularly during CLIP training, positively impacts the model representation quality' might be too strong of a claim. As explored in prior work ("Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)"), language supervision might not be the sole reason for model generalization. There're multiple variance between VLM and other modality and author should not attribute such improvement solely on language supervision.

Author response:
Thank you for your valuable feedback. I appreciate the time and effort you have put into reviewing my work. Below, I have addressed each of your points in detail.

**Point1** : We acknowledge that, as already mentioned in the paper (first paragraph in the introduction), some prior work such as [1] suggested that the model generalization is indeed influenced by multiple factors beyond language supervision alone.  
 However, we note that there are certain shortcomings in assessing the role of language in generalization of CLIP models in [1]. Specifically, the dataset (ImageNet-Captions) that is used in [1] to investigate the effect of language in domain robustness could lack domain information (e.g. painting, sketch, clip art, etc.) in the captions that are used for training and thus does not utilize the language capability to tackle domain shift. This stems from the fact that in this dataset, the captions are taken from the original text data of the Flickr images. Hence, the language is not used in its maximum capacity to train the model. However, in our case, we expect all the composition constituents, which are attributes and objects, been described numerous times *individually* in the training data. In fact, we investigate compositional OoD generalization, which assumes that the novel combination of *known* concepts appear at the test time and evaluate the role of language in this type of OoD generalization. Aside from this, ImageNet-Captions is orders of magnitude smaller (~460k images) than the datasets that are used to train CLIPs, which are often in the order of 100 millions or few billions of images, which could be insufficient to expect the language to play a significant role in enhancing the generalization. On the other hand, we benchmark CLIPs that are trained with much larger training sets. Therefore, our work makes a better setup to assess the role of language in the generalization. 
 Therefore, one of our aims was to re-assess whether the language could bring any additional value to the table beyond what has already been known in the  literature. 

We have to emphasize that our intent was not to attribute the observed improvement in model performance solely to language supervision, but rather to see if the language could be *one of the* the  drivers of improvement in our compositional generalization setting. To make this happen, we conducted *controlled* experiments  to specifically isolate and understand the impact of language factors on model generalization. For instance, we compared models with *identical structures* and *training methodologies*, with the only variable being their *training datasets*, such as the CC and YFCC models. Both models employ the same image and text encoders, yet they yield different outcomes. Further analysis revealed that the captions in the CC dataset are of markedly higher quality (table 1), which provides empirical evidence of the language factor's influence on model performance.

We believe these controlled comparisons are essential to discern the discrete contributions of different variables to model generalization. In future iterations of our manuscript, we will strive to more precisely articulate the multifaceted nature of model generalization, ensuring that the significance of language supervision is presented as one of several contributing factors, rather than the sole determinant.

We hope this clarification addresses your concerns, and we are committed to revising our manuscript to reflect a more nuanced understanding of the elements that contribute to the robustness of model generalization.

[1] Fang, Alex, et al. "Data determines distributional robustness in contrastive language image pre-training (clip)." International Conference on Machine Learning. PMLR, 2022.

Reviewer comment (Experiments):
Conclusion are less convincing due to the limited candidate in each experiment. For instance, in Table 1, it will be interesting to shows the NMI for a subset of LAION with the same number of data to other dataset. Also in Table 2, there's only 4 results, please consider adding more variance of dataset and CLIP architecture.

Author response:
**Point2** : Table 1 in Section 3.3.1 now includes NMI values for various subsets of the LAION dataset. It is worth noting that these different LAION subsets originate from the same source but undergo different filtering processes. The filtering appears to alter the token distribution, which in turn affects the NMI values. However, NMI has no direct relationship to the dataset size. For instance, as the results indicate, despite its smaller size, LAION 400m shows a bit higher composition diversity compared to that the LAION 2B.

**Table1: Normalized Mutual Information between the attributes and objects calculated for captions of some CLIP training sets. The domain of these random variables are defined based on the compositions present in our generated dataset.**

| **Dataset** | **Dataset Size** | **NMI**  |
|-------------|------------------|----------|
| YFCC        | 15m              | 0.9390   |
| CC          | 12m              | 0.8903   |
| LAION       | 400m             | 0.8307   |
| LAION       | 12m              | 0.9024   |
| LAION       | 2B               | 0.8541   |

Additionally, Section 4.1 now contains expanded results, including additional models in the updated version of Table 2.

**Table2: Number of common dimensions across factors and switching dimensions for color manipulation in 3D Shapes dataset**

| **Dataset** | **Architecture** | **OoD Accuracy** | **# of Common Dims** | **# of Switching Dims** |
|-------------|------------------|------------------|----------------------|-------------------------|
| LAION       | ViT-L/14         | 43.14%           | 2                    | 40                      |
| LAION       | ViT-B/16         | 38.89%           | 5                    | 60                      |
| LAION       | ViT-B/32         | 32.84%           | 7                    | 90                      |
| OpenAI      | ViT-L/14         | 39.91%           | 3                    | 5                       |
| OpenAI      | ViT-B/16         | 37.73%           | 4                    | 10                      |
| OpenAI      | ViT-B/32         | 33.43%           | 6                    | 30                      |
| CC          | RN50             | 9.64%            | 15                   | 200                     |
| YFCC        | RN50             | 3.60%            | 21                   | 250                     |

Reviewer comment (Writing):
The narrative after section 4 is a bit too rush, it's hard to follow the method and results. For instance, what is 'dimensions' in 4.1 stands for? And more context over 'switching dimension' would be helpful. Moreover, I cannot tell how the conclusion of 'A higher level of accuracy in the image retrieval task indicates that the model embeddings are more disentangled.' can be drawn from experiment in 4.2.

Author response:
**Point 3**: In Section 4.1, when we refer to 'dimensions,' we are speaking of the individual elements within the embedding vector generated by the CLIP model. To clarify the concept of 'switching dimensions,' we conducted experiments where these individual elements are systematically altered to assess their impact on the performance of the model. By switching, we mean replacing the values in these dimensions by those of the samples that have a different level of attributes.   This process helps us understand which dimensions of the embedding vector correspond to specific attributes in the data.

Regarding the image retrieval task discussed in Section 4.2, we propose that a more decomposable and disentangled representation leads to better retrieval performance. This is because when the embedding of an image is decomposable, adding the embedding of an attribute (like a color or shape) to the image’s embedding results in an alteration of only the intended attribute in the resultant image, without affecting other characteristics. Conversely, if the embedding is entangled, the same operation would inadvertently alter the entire image, not just the intended attribute. Therefore, the ability to perform precise modifications through embedding manipulation is indicative of a disentangled representation, which, in turn, suggests higher accuracy in retrieval tasks.

In our additional experiment using the 3D Shapes dataset, we aim to correlate the model's disentanglement metrics with its Out-of-Distribution (OoD) generalization performance. We employ a classifier trained to recognize common features across different images, such as color. By identifying which dimensions of the embedding vector are significant for such features, we gain insights into the disentanglement of the embeddings. Our findings suggest that in higher-performing CLIP models on OoD benchmarks, fewer dimensions need to be switched to achieve an embedding that aligns with a modified attribute, thus reinforcing the connection between disentanglement and OoD performance.

We have included additional diagrams in the revised manuscript to visually demonstrate the processes and outcomes of these experiments, specifically experiment 4.1, for greater clarity.

We hope that this explanation resolves the ambiguities and look forward to further comments and suggestions.