Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper pays attention to analyzing the perceptiveness of multi-modal classifiers, proposing a perceptual score, which is very helpful for the community. Using the proposed perceptual score, the authors analyze a wide range of models on different datasets, finding that multi-modal models tend to ignore some modalities while taking shortcuts. In terms of bias, this paper finds that SocialIQ is biased by sentiment. The above interesting findings are helpful for researchers to propose new models and develop new datasets.
main_review: The proposed perceptual score is simple and can be easily computed, which could be a useful metric to measure the perceptiveness of multi-modal classifiers. The perceptual score is novel to me. The analysis is comprehensive, including many existing models on several multi-modal datasets. And the findings are inspiring, which is helpful for the community.
limitations_and_societal_impact: 1. some models that use more information should be analyzed, such as Oscar [1].

2. this paper only analysis the vision-language datasets, but there are some datasets composed of other modalities (vision, audio and language).

[1]. X. Li et al.. Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. ECCV, 2020.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper proposes a measure for evaluating multimodal models on how much they take into account the various modalities in making their predictions. The proposed "perceptual score" evaluates a model's performance on each example when one of its modalities is randomly sampled from other items in the dataset (an expectation of performance when paired with all possible versions of the modality being tested). The general idea is that if the performance when paired with a random sample is similar to the performance when paired with the ground truth modality, then the model is likely not taking into account that modality when making its prediction.

The paper evaluates the metric on several vision and language tasks (VQA, VQA-CP, SocialIQ, and VisDial) with several interesting findings (e.g., that one can achieve relatively high performance on SocialIQ without even training a model and only considering one of the modalities).
main_review: Originality: This is my main concern with the paper and why I am giving a score of 5 in my initial review. Hessel and Lee (EMNLP 2020, https://aclanthology.org/2020.emnlp-main.62.pdf) propose a very similar measure for the same problem studied in this paper, called EMAP. The intuition of the two measures is very similar: measuring the performance on a random sample of modalities. I didn't have time to compare both papers exactly, but I would encourage the authors of this paper to not only cite Hessel and Lee 2020, but include space to compare the measures (since the EMNLP paper has been published for a while now, I would expect the authors of this paper to know about it, and am surprised it was not cited). At a surface level, it does seem the measures are slightly different; for example, they compute a single metric for a dataset while this paper proposes to provide metrics for each modality independently (although I am not sure if this is just \hat{f}_T and \hat{f}_V in Eq 3 of Hessel and Lee 2020). The paper in submission also includes discussion of some normalization used to compare models (and tasks?) directly, and additional analysis on several datasets. 

I would expect a final version of this paper to include a discussion of how the method proposed in this paper compares with the measure introduced by Hessel and Lee 2020. Even if the measures are equivalent, this paper does offer additional analysis and variants of the measure that may be interesting standing on their own, although it may require significant reframing of the paper.

Quality: I would have liked a more theoretical intuition of the normalization methods (3.2.2). I was also somewhat confused about when you would use the two different kinds of normalization or how to combine them.

The experiments and analysis are quite thorough, and the finding on SocialIQ is very interesting.

Nit: in lines 160--161, "If the accuracy doesn't change, irrespective of whether modality M_m is available or not, the model f *doesn't* perceive the modality M_m" while this is very *likely* the case, I don't think we can conclude that it doesn't perceive the modality at all; after all the metric is an estimate only (expectation over items in the dataset).

Clarity: The paper is easy to follow and understand for the most part. Being more clear about what "bias" means here could be useful, since this term is ambiguous. 

Smaller things:
- Table 1, may be useful to have a reminder of whether a high or low perceptual score is good.
- Table 3: since Acc_D is not dependent on the models maybe don't have different numbers in each model row, rather just have its own combined row. 

Significance: A measure like this could be useful for researchers working on multimodal problems. The techniques used in this paper to analyze SocialIQ and VQA-CP provide a good framework for people to interpret and deeply analyze the perceptual scores and their implications on a model and dataset. I worry though that there is significant overlap with EMAP that will cause confusion between the two measures, so would appreciate the authors clearing this up.
limitations_and_societal_impact: Although brief, I appreciated the discussion of a "false sense of security" these measures will provide. I have similar concerns that these kinds of metrics will be taken as fact (e.g., that high perceptual scores mean a model/dataset don't have any (inductive) bias).
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper aims to quantify the degree to which a model relies on the different subsets of the input features (i.e., modalities), and introduces the perceptual score. In experimental, they find that state-of-the-art multi-modal models for visual question-answering or visual dialog tend to perceive the visual data less than their predecessors.
main_review: 1.	The topic to explore the importance of different modalities is interesting and the reviewer thinks that it will bring more interpretability for multi-modal learning
2.	The proposed method is direct and easy to follow. And the experiments find that multi-modal models tend to ignore some modalities while taking shortcuts.

limitations_and_societal_impact: Weakness:
1.	This manuscript is more like an experimental discovery paper, and the proposed method is similar to the traditional removal method, i.e., traverse all the modal feature subsets and calculate the perceptual score, removing the last subset. The reviewer believes that the contribution of the manuscript still has room for improvement.
2.	The contribution of different modalities of different instances may be different, e.g., we have modalities A and B, some instances with good performance of modality A  which belongs to the strong modality, whereas some instances with good performance modality B which belongs to the strong modality. Equation 3 directly removes the modal subset of all instances. How to deal with the problem mentioned above.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper proposes a score to understand biases in datasets and models by trying to assess the reliance of a model on different modalities. Specifically, this perceptual score is defined as the difference between the model’s accuracy with all modalities and with one modality removed. The proposed method is  used to bring insights into significance of text and visual modalities for different question answering systems. 
main_review: The paper proposes a method to understand deep neural network models. In particular, multimodal systems are often hard to interpret and the significance of different modalities in solving the task at hand is often not well understood. This is clearly an important problem and in that the paper is clearly well motivated in attempting to do that. 
The paper is clearly written and mostly easy to follow. 

The current definition of perceptual score and its role in understanding the significance of different modalities as well as biases in models is not very convincing. 

1. The perceptual score is showing how the overall accuracy is changing. However, its unclear what this means at sample level. How do we interpret prediction label flip (from correct to incorrect as well as incorrect to correct)  at sample level when “removing” a modality. 


2. The modalities are not really removed and I agree that it would be difficult to do for most systems. However, randomly swapping the modality from another data point in the test set is not convincing as well. How do we understand this randomness or stochastic nature of the perceptual score ? Moreover, using the modality input from another data point biases it towards the dataset again. Wouldn’t it be better to just use random “noise” to remove a modality -- say for example some sort of gaussian-noise image input to “remove”. Can this work ? One can also use random image or text from a different dataset. 


3. It is unclear how  including Z_D as a normalizing factor helps include the difficulty of a task in the perceptual. Can the authors clarify how  normalization by this factor is really helpful in grounding the perceptual score. If the training data consists of almost even distributed classes then \hat{Acc}_D is basically the 1/#classes which can be large or small depending on the number of classes. 


4. The empirical analysis in the paper has focused on using the metric to draw insights into different tasks and systems. However, some empirical analysis on why the metric itself makes sense is necessary ? How do we understand that score of this form is a reliable way to understand datasets and models ?  The paper does not compare or even relate the proposed perceptual score directly to any prior methods which makes it even harder to understand its significance. 


5. I am not able to fully understand how the perceptual score aids the bias analysis of CSS. From Table 2, it does look like that CSS is achieving better performance by balancing class priors which are changing from train to test.  But what is the role of perceptual score here ? The numbers in Table 2 and conclusions from it can be drawn without the perceptual score.   Do we need the perceptual score to do this analysis ? 


limitations_and_societal_impact: The paper proposes methods to help understand deep multi-modal models. The authors themselves have outlined two limitations in their conclusions, although they have not been discussed in detail.
review_title: null

Meta-review: Understanding how sensitive multi-modal models are to their various inputs is an important topic and this submission proposes a straight-forward to estimate these perceptual sensitivities. The proposed method is demonstrated in a couple of domains and the author response extended to an additional setting. Reviewers unanimously recommend acceptance and the AC concurs. Authors are strongly encouraged to include the additional audio setting in the camera ready and to clarify the comparison to EMAP.
Decision: Accept (Poster)