Paper: Towards Explainable and Efficient Multi-Modality Learning: Domain-Agnostic Concept Space Paired with Domain-Specific Projection Models (ICLR-2024)

Reviewer comment (Writing):
First, a major weakness is the lack of a clear and rigorous definition of the main notions of the paper. For instance: what is the difference between a domain and a modality? What is a concept and how are they defined?

Author response:
We have realized our uses of modality and domain may cause confusion among readers. To clarify, we have added a footnote at page 3 of our paper to provide definitions of modality and domain and modified the mixed uses at various places in the paper. In our work, we define modality as a medium such as vision and natural languages whereas domain is defined as a specific representation within a modality. Moreover, a single modality could contain more than one domain such as different languages (domain) within the set of natural language (modality). Extending the example of languages, our framework is capable of connecting information from  two languages such as English and Spanish in the transparent concept space by using an English-specific projection model and a Spanish-specific projection model, which would allow cross-domain tasks such as machine translation to be carried out in an explainable manner.

Reviewer comment (Novelty):
Since the aim of the paper is to learn a kind of universal representation, some important references are missing such as the works of Li et al in the few shot domain (see for instance [these papers](https://github.com/VICO-UoE/URL)) or others references on the notion and evaluation of universal representations. Moreover, since one of the targeted objectives is also to gain transparency and interpretability, I think that positioning of the contribution compared to the concept-based approaches in the XAI field is also missing (see for instance [this paper](https://arxiv.org/abs/2205.15612)).

Author response:
After examining the reviewer’s suggestions on related work, we agree that they are closely related to the aforementioned literature in our paper as well as our proposed framework. We have incorporated [1] under the Multi-Modality Learning section and [2] under the Concept Learning section in our Related Work discussions.

Reviewer comment (Theory):
Some technical choices are not motivated and justified. For example, why a softplus function in equation 1? Why a KL divergence? Moreover, it is not clear how the proposed concept space learning approach is novel compared to the line of works on box embedding (see for instance the references used [here](https://www.iesl.cs.umass.edu/box-embeddings/main/index.html)). Q is not clearly defined in Equation 3. Globally, the formalization lacks some rigors. In the cross modality joint training, why not weighted loss terms?

Author response:
>Why is a Softplus function used in Eq. 1 and what is the novelty of our proposed framework when compared to existing works on box embeddings?

In our implementation, we have chosen to adopt a box embedding space proposed by [3] to organize our concept space so we followed the same definitions for calculating probabilities in the concept space which are proved by Li. et al. In short, a Softplus function relaxes the constraints of boxes by allowing a valid joint probability to be calculated for two disjoint boxes. We did not include specific details about this proof as we believe it is not our major contribution. We argue that the idea of a framework that leverages an explainable domain-agnostic concept space paired with domain-specific projection models is our work’s major contribution. In other words, we view our use of the box embedding space as an implementation choice. We hope to see better ways of organizing an explainable concept space being proposed in the future and our framework will be adjusted to incorporate those methods. 

> Why is a KL divergence loss used to pretrain the concept space and what is the definition of Q?

During pre-training, our goal is to produce a concept space that accurately reflects real-world concept relationships. In other words, the concept space should produce probability distributions of concepts that are as close as possible to the ground truth probability distributions. So we think it is natural to use a KL divergence as the concept space loss which measures the difference between two probability distributions and the concept space should be optimized to minimize this difference. As for the use of symbol Q, we have followed the common definition of KL divergence where P denotes observations and Q denotes a model’s approximations of P [4]. In our paper, a more formal definition of Q can also be found roughly four lines above Eq. 3.

We have also updated our Eq. 6 of the overall loss to add weights to projection losses and joint losses.

Reviewer comment (Experiments):
No state-of-the art comparison is given for the zero-shot image-text matching and more globally the experimental study lacks details.

Author response:
> Comments regarding the image-text matching experiment:

We agree with the reviewer that a more comprehensive comparison between the state-of-the-art and our results would be beneficial. However, as the CLEVR dataset is not traditionally used as an image-text matching dataset, there seems to be a limited number of works that provide comparable results. Additionally, we view the image-text matching task as one of many downstream tasks that our framework can be applied to so this experiment serves more as a warmup and introduction to the application of our framework. But we will keep this shortcoming in mind for our future iterations of this work.

Reviewer comment (Evaluation):
The paper provides no evaluation of the interpretable part of the approach. Evaluating interpretability is difficult but this point should be more discussed in the paper.

Author response:
As for the suggested evaluation of interpretability, we appreciate the reviewer’s recognition that interpretability is hard to measure. However, we have expanded our paper to include a Discussion section where we approach the evaluation of interpretability by discussing a benefit of having an interpretable concept space: the ability to address hidden bias in learned knowledge.

Reviewer comment (Experiments):
One strong hypothesis of the proposed work is to have annotated data in order to replicate real-world concept entailment probabilities. This is a big constraint and this kind of annotated data is often unavailable. How to update the approach in order to tackle this issue? Moreover, what kind of relations between concepts are targeted in the proposed approach? (semantic ones, hierarchical ones). Since using geometric-based embedding is a way to preserve semantic or structural relations, this point should be more discussed in the paper and in the evaluation.

Author response:
- Q1: 

    We think this question is very insightful and interesting and we have added a discussion around how to obtain ground truth concept entailment data in our paper. In short, we believe our approach of calculating concept entailment probabilities using datasets’ labels can be adapted to more comprehensive datasets with a greater number of concepts/labels. Additionally, we view the rich textual data that is widely available today as a valuable resource to extract concept relations as demonstrated in [5]. As for the specific type of concept relationship, in this paper we mainly focus on the denotational relationship between concepts but we aim to investigate the feasibility of extending support to more complicated concept relationships in our future works.

Reviewer comment (Experiments):
In the experimental work, what is the effect of the size of the concept domain space? More globally, what are the implicit or explicit constraints on the concepts in this concept space?

Author response:
- Q2:

    During trials of our experiments, we find setting the dimension to 50 yields the best results. Increasing the size of the concept space would lead to slower convergence and since the calculation of probability involves volumes of boxes, a larger dimension could expose our framework to overflow issues. In terms of constraints, our concept space poses minimal restrictions on concepts other than the type of relationships it supports.

Reviewer comment (Theory):
What about the disagreement between modality in the proposed approach?

Author response:
- Q3:

    While our current training method relies on accurate cross-domain data to produce valid projection models, we believe tackling disagreement between modalities is an exciting challenge that offers many future research opportunities. One possible solution we could think of is that the projection models can be trained on a small set of verified training data, which could be achieved thanks to their quick convergence as shown in our ablation study. These trained projection models can then be used to detect disagreement between domains similar to how our framework handles the Image-Text Matching task.

Reviewer comment (Theory):
In the formulation, although the paper states in the introduction that the “concept space is optimized to reflect real-world relations between concepts via entailment probabilities,“ it only considers probabilities where concepts appear together in the training data. This may be unrealistic from an OOD generalization perspective; for example, if the pair “(blue, sphere)” is missing from the training set, it will be assigned a zero probability during inference. This could imply that the model may not generalize well to unseen data, specifically unseen concept pairs, making it unsuitable for real-world settings where concepts not seen together in training data may occur together.

Author response:
Comments regarding the formulation of the concept space learning goal:

We agree with the reviewer that the ability to generalize to Out-of-Distribution (OoD) samples, in our case concept pairs, is paramount to any ML system with a goal of being deployed in the wild. We also hold the same opinion as the reviewer that real-world settings may contain concept pairs that are rare but valid. However, as there are more questions than answers in the emerging research direction of concept-centric learning, we deliberately chose CLEVR dataset to base our initial investigation on for the dataset’s highly controlled mini-world which offers clear concept relations without the existence of OoD concept pairs. But in future iterations of our work, we do believe the investigation of how to improve our framework’s OoD Generalization ability could lead to many exciting research projects. One possible solution that we could think of right now is to continuously update the concept space which would not cause too much burden as it is quick to train thanks to the modest size of its parameters. Other possible solutions could include popular methods under the realm of Active Learning.

Reviewer comment (Experiments):
The paper compares its method with older methods that do not represent the state-of-the-art in VQA on CLEVR. The baselines should be more comprehensive and contemporary. An additional baseline worth including is using the pre-trained CLIP model as-is in the VQA pipeline, since CLIP pre-training aligns vision-language pairs in a similar way.

Author response:
As for the reviewer’s comments regarding the state-of-the-art on CLEVR VQA task, in our original table, we have included the work by Yi et al [1] which currently holds the first place for this task. However, after evaluating the suggestion, we have included a more recent work by [2] which did not appear in our first version as this work does not rely on extensive program annotations as other works including ours does. To address this difference and maintain fairness, we have added a note regarding the performance of [2].

Reviewer comment (Experiments):
It is unclear whether the validation set contained concept pairs not occurring in training, how many such pairs there were, and whether the model performed better than random guessing on those validation samples.

Author response:
Comments regarding the validation set:

As mentioned previously, we agree with the reviewer’s point that the ability of generalizing to unseen samples is important to any ML systems. But as stated in the paper, we follow the same train-validation split in the original CLEVR dataset which does not include OoD concept paris.

Reviewer comment (Theory):
The loss function in Eqn 3 scales quadratically with the number of concepts, which may not be scalable for real-world datasets like ImageNet.

Author response:
Comment regarding the scalability:

We acknowledge that efficiency is one of the limitations of our current work and we hope future iterations of our work could improve upon the current one. But we would like to add that the training time of our concept space is very modest compared to months of training time on powerful servers for today’s large ML models.

Reviewer comment (Writing):
The distinction between “domain” and “modality” is unclear in the text. It is not evident whether these terms are interchangeable or have separate meanings.

Author response:
We have realized our uses of modality and domain may cause confusion among readers. To clarify, we have added a footnote at page 3 of our paper to provide definitions of modality and domain and modified the mixed uses at various places in the paper. In our work, we define modality as a medium such as vision and natural languages whereas domain is defined as a specific representation within a modality. Moreover, a single modality could contain more than one domain such as different languages (domain) within the set of natural language (modality). Extending the example of languages, our framework is capable of connecting information from  two languages such as English and Spanish in the transparent concept space by using an English-specific projection model and a Spanish-specific projection model, which would allow cross-domain tasks such as machine translation to be carried out in an explainable manner.

Reviewer comment (Experiments):
The paper states that it uses a neuro-symbolic-inspired approach in VQA, but this is not mentioned elsewhere, and no neuro-symbolic baselines are provided.

Author response:
Comments regarding the VQA experiment:

In the evaluation of our framework’s performance on CLEVR VQA task, we follow a popular approach of adopting a set of neuro-symbolic programs initially proposed by Johnson et al. [7] and used by [8-11]. As stated in the paper in Sec. 4.2, we implement these neuro-symbolic programs as deterministic functions which follow the same definitions from the [7] and a program generator is used to produce a set of programs corresponding to a question. In other words, when adapting to the downstream task of CLEVR VQA, our framework is only exposed to the vision modality to conduct probabilistic reasoning. So we think that a CLIP-like model might not be the best candidate for such a task.

Reviewer comment (Writing):
The meaning of some symbols is unspecified and left for the reader to infer (e.g., Section 3.1.2: f_A: A -> C; what does ‘A’ mean here?).

Author response:
We agree with the reviewer that our use of the symbol “A” in $f_A: A \rightarrow C$ at the beginning of Section 3.1.2 may cause some confusion and we have added a note regarding A’s denotation at the end of the sentence. Here, we simply intend to use “A” as a general way of denoting a domain.

Reviewer comment (Writing):
The paper contains grammatical errors in several places (e.g., Introduction: “has also drew criticism”; Program generator: “is freezed”).

Author response:
And we appreciate the reviewer pointing out a few grammatical errors that slipped into our paper’s final version which all have now been corrected.