Perceptual Score: What Data Modalities Does Your Model Perceive?
Authors: Itai Gat
Idan Schwartz
Alex Schwing
Venue: NeurIPS 2021 Poster (2021)
Keywords: VQA
Visual Question Answering
VQA-CP
Perceptiveness
Multimodal
Evaluation
Visual dialog
SocialIQ

Abstract: Machine learning advances in the last decade have relied significantly on large-scale datasets that continue to grow in size. Increasingly, those datasets also contain different data modalities. However, large multi-modal datasets are hard to annotate, and annotations may contain biases that we are often unaware of. Deep-net-based classifiers, in turn, are prone to exploit those biases and to find shortcuts. To study and quantify this concern, we introduce the perceptual score, a metric that assesses the degree to which a model relies on the different subsets of the input features, i.e., modalities. Using the perceptual score, we find a surprisingly consistent trend across four popular datasets: recent, more accurate state-of-the-art multi-modal models for visual question-answering or visual dialog tend to perceive the visual data less than their predecessors. This is concerning as answers are hence increasingly inferred from textual cues only. Using the perceptual score also helps to analyze model biases by decomposing the score into data subset contributions. We hope to spur a discussion on the perceptiveness of multi-modal models and also hope to encourage the community working on multi-modal classifiers to start quantifying perceptiveness via the proposed perceptual score.  

Perceptual Score: What Data Modalities
Does Your Model Perceive?
Itai Gat
Idan Schwartz
Alexander Schwing University of Illinois at Urbana- Chhaaa-Champaign
Technion
Technion NetApp
Abstract
Machine learning advances in the last decade have relied significantly on large- scale datasets that continue to grow in size. Increasingly, those datasets also contain different data modalities. However, large multi-modal datasets are hard to annotate, and annotations may contain biases that we are often unaware of. Deep-net- bes-eedd classifiers, in turn, prone exploit those biases and to find shorteuts. To study and quantify this concern, we introduce the perceptual score, metric that assesses the degree to which model relies on the different subsets of the input features, ie., modalities. Using the perceptual score, we find a surprisingly consistent trend across four popular datasets: recent, more accurate state-of-the-are multi-modal models for visual question-answering or visual dialog tend to perceive the visual data less than their predecessors. This trend is concerning answers hence increasingly inferred from textual cues only. Using the perceptual score also helps analyze model biases by decomposing the score into data subset contributions. We hope to spur a discussion on the perceptiveness of multi-modal models and also hope to encourage the community working on multi-mode classifiers start quantifying perceptiveness via the proposed perceptual score.
Introduction
Machine learning advances over the last decade are remarkable. Challenges that seemed daunting merely ten years ago now breeze, and new applications that barely dared to dream about seem achievable within the next few years. Indeed, accuracy metrics on tasks like visual question answering and reasoning suggest significant improvements.
Reported improvements are large extent due to the availability of large datasets [1 [1-3], com- putational performance advances, e.g., for GPUs, and a better understanding about how to encode inductive biases into deep-nets, e.g., by using rectified linear units [4], normalization [5], skip con- nections [6], transformers [7], etc. However, importantly, developed deep-net architectures are not guaranteed to solve a given task. There a chance that they may instead exploit dataset biases.
This concern is surely in part due to non-robust training techniques, and plethora of methods improve classifier robustness [8-10]. However, datasets play an important role in controlling the extracted bias as well. For instance, if correct answers in question-answering task are significantly shorter than incorrect ones, classifier training should use answer length as a cue. Although this seems reasonable, for audio-visual scene aware dialog, Schwartz et al. [11] find for example that in many cases the question alone sufficient generate scene-aware dialog response, avoiding the need to look at the video. Hence, in order to assess the suitability of classifier, we need to understand how much relies on different data modalities.
To quantify how much classifier relies on its different input modalities, we introduce the perceptual score. The perceptual assesses the degree which a model relies on a modality. To do the perceptual score permutes the features of modality across samples in the test set after the classifier
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Permuned
Permul
Accuracy
Parcappial
80
Image Perceptual Score
Question Perceptual Score
How many animale are pictured?
How animale pictured??
60
Percaptial
Question
40
How many animals pictured?
Why are the men jumping? What color the ground?
What oolor the car?
LMH (2019)
RMFE (2020)
CSS (2020)
(a) Visual question answering (VQA) data. Modality perceptiveness is measured through a permutation test.
(b) Progress of VQA-CP models.
Figure I: Multi-modal datasets often have undesired biases: (a) To identify those biases we suggest the perceptual score new metric. It assesses the change in prediction when model's s input some modalities is permuted during testing. the classifier output remains identical despite permutation, model doesn't 'tt perceive the modality. (b) Using the perceptual score we identify that recent progress of VQA models may not be entirely due to better reasoning.
was trained, illustrated in Fig. la. If the classifier's performance drops below chance level, the perceptual score is high. This intuitively applies to single-mon modaalty models too: randomly permuting test data and labels after training results in chance-level classification accuracy.
Using the perceptual score, we find surprisingly consistent trend across five popular datasets (VQA, VQA-CP, VisDial, SocialIQ, DISCO): recent, more accurate state-of- the-art multi-modal models for visual question- answering or visual dialog tend to perceive the visual data less than their predecessors (see Fig. 1b). This trend concerning only. Using the perceptual score also helps to analyze model biases by decomposing the score into data subset contributions. For example, the perception of image and question varies depending on the question type. None of the recent VQA-CP models showed high image perception scores for 'number' type questions. A surprisingly low image perception score obtained for the state-of-the-art model when confronted with 'yes/no' questions.
answers hence increasingly inferred from textual cues
We hope the perceptual score spurs discussion regarding the perceptiveness of multi-modal models classifiers to start quantifying
and we also hope to encourage the community working on perceptiveness of models.
Our contributions:
We propose the perceptual score, simple yet effective method for assessing the perceptive- ness of multi-modal models towards a modality.
Our experiments span multiple datasets and models. We find that multi-modal models tend to ignore some modalities while taking shortcuts.
Subsequently, we investigate the sources of bias on popular multi- datasets such VQA-CP and SociallQ: We find that SocialIQ is biased by sentiment, and bias in the VQA-CP model results from shifting the training priors to more closely resemble those in the test set.
2 Related Work
The perceptual score assesses the degree to which classifier relies on a particular input modality. This is related to studying datasets and their biases, methods which aim to reduce the biases captured by classifiers and work which studies the importance of features. We review all three areas next.
Datasets and bias: Data has been central element for machine learning progress [12-14] in the last two to three decades. The ImageNet challenge [2] and the development of AlexNet [15] sparked the deep learning era. But as datasets grow, biases emerge which may go undetected for long time. For instance, the background ImageNet can reveal information about the object class [16]. Also, with the increasing popularity of crowdsourcing systems like Amazon Mechanical Turk, many datasets

---

are annotated in uncontrolled environments. Different annotators are hence injecting unknown socio-economic properties into dataset annotations [17]. Those dataset biases can be detrimental to the considered task. For instance, meticulously collected visual question answering (VQA) data [1] aims to provide platform for exciting research to advance image-language understanding. However, it is non-trivial to remove biases from this type of data. Indeed, it was reported that the question solely is sufficient to detect the correct answer [18, 19], i.e., no image information is required. In an attempt to fix this bias, the dataset was re-annotated [20]. or the train and test split were re- dialog (AVSD) [22], the question cue is often stronger, making the desired video and sound reasoning implausible or unnecessary. Likewise, SNLI [23], aims to determine the correctness of a hypothesis given premise. However, Gururangan al. [24] point out that an internal bias exists: linguistic features not related to the premise correlate with the label. Further, the Story Cloze [25] dataset permits develop models which estimate the correct ending of story. Schwartz al. [26] show that, for this dataset, length alone is a powerful feature to determine the correct ending. Teney et al.
organized [21]. Similarly, Schwartz et al. [11] reported that in audio-visual-scene-ane
[27] discuss the pitfalls of current duttrf-distribution VQA methods. Perceptual score provide way of identifying and quantifying these limitations.
Recently, Zadeh et al. [28] proposed SocialIQ, a dataset intended to reason about social situations in videos, specifically, emotion detection in a social situation. In this dataset, given a video and a question, the correct social situation should be recognized, e.g., "The man is upset because he is being insulted." We show that it is easy to pick the correct statement using only the text data. This is possible because the statement's 's correctness often correlates with the statement's 's sentiment. Again, much like for VQA and AVSD, image information doesn't seem to be necessary for reasonably accurate performance. Hence, biases exist which permit to 'address the dataset' without addressing the task. For SociallQ, we attribute these biases to the fact that social reasoning is considered difficult, even for humans. Hence an annotator's expertise is particularly important.
Importantly, going forward, we think it is elusive that we will be able to create unbiased datasets. We hence need techniques to automatically measure biases. In this work, we provide mechanism that permits to do this for any dataset.
Methods to reduce bias: Several methods have been suggested to reduce bias from a dataset. Some techniques require prior knowledge of the biased variables, for instance, gender bias in vision is addressed by masking related features (e.g., faces) [29-38]. Also, some techniques require access to the test set to re-balance [39]. Additionally, various approaches were proposed for visual question answering [40-43]. These methods use classifier trained only on the question modality to regularize bias directly. Cadene et al. [40] suggest to mask a model's softmax prediction with softmax prediction of subset classifier trained on the question modality. Clark et al. [41] use an ensemble method of a full classifier paired with a question subset classifier using products of experts [44]. Common to all those approaches is the use of a subset classifier to indicate reliance on subset of the data. In contrast, we assess the perceptiveness of the original model based on permutations of subset of the data.
Feature importance methods: Also relevant to our work are techniques that measure feature- importance. These works generally focus on understanding why model made particular prediction. To this end, Ribeiro et al. [45] introduce LIME, a local explanation method that approximates a linear explanation model around given example. Later, Shrikumar et al. [46] proposed DeepLift specifically for deep net models. Lundberg and Lee [47] introduced SHapley Additive exPlanations (SHAP), a method to estimate feature importance using an approximation of the features Shapley values. LIME assumes that the local model is linear while SHAP does not have any such assumptions. Lundberg and Lee [47] show a connection between DeepLIFT and Shapely values introduced as 'Deep SHAP. Each of those techniques concentrates on the importance of features, whereas we focus on the importance of a modality as whole. Hessel al. [48] proposed to assess how much classifier benefits from interactions across different data modalities. To do this, EMAP approximates a multi-modal classifier with classifiers that each depend only single data modality. In contrast, measure the perceptiveness of each modality separately and for every data point independently, i.e., we aim to identify how much classifier relies on each data modality.
3

---

Test set
Predictions
Predictions
(M)
Acce (M) (Maa)
D
(Ma)(Am)
(a) Three modalities data.
(b) Calculation of sample perceptual score.
Figure 2: Calculation of sample perceptual score: (a) Shows three-modalin test set where each sample is colored differently. A different line style distinguishes modalities. (b) Demonstrates how to compute the sample perceptual score. Given test set D, we measure the sample x accuracy of the original model on the left. We calculate the average accuracy across several permuted samples on the right (see Eq. (3)). The perceptual sample score is the difference of those two terms (see Eq. (2)).
The Perceptual Score
The perceptual score quantifies the degree to which model relies on the input data subset thereof. Said differently, the perceptual score assesses the importance of data or subset of the data for model's 's result. For instance, if model answers questions about image without using extracted from the image modality, the model does not perceive the image modality. In this case we want the perceptual score for the image modality to be lower than that of a model which relies heavily on the image modality.
We believe that reporting the perceptual score of model in addition to its accuracy is particularly important in multi-modal setting. As community we are developing increasingly complex models. However, to date know very little about what parts of the data these models rely We think this lack in our understanding is due to a missing easy way to quantify how much a model relies on available data modalities. To rectify this we hope to illustrate that simple yet intuitive score like the proposed perceptual score is very useful and easy to report too.
The following sections describe the perceptual score of a modality for a given model and dataset.
3.1 Setup
Let D { v1)..... (Zip) 3(p denote the test set data, where [D] refers to the number of samples in the dataset. Each sample is pair consisting of the input data E X' and its corresponding label y1 y. Here, y denotes finite set of possible classes. In multi-modal datasets which we consider here, the data I; can be separated naturally into different parts as illustrated in Fig. 2a. For instance, in the SociallQ task, we can partition the data into video-related, question-relater and answer-related parts. Formally, let M {M,...., MM } be set of modalities of size [M], e.g.. the video-, the question- and the andwer-modality. We partition the data x; into its modalities using ! M2) fx M. MM
set notation, ie.,, [i:M, ,...,.". Th MMMi }. We use x; modalities, i.e., the superscript can set.
} to refer to the first two
3.2 Perceptual Score of a Data Modality
The perceptual score P),(Mm) of a model f: X' + y towards modality Mm on data D is defined as
(M.D(mm)= Z (TL (Pra))
(1)
i.., the normalized expectation of sample perceptual scores Pf.z,1 (Mm). (M.. Here, Z indicates the normalization factor, which can either be determined by the dataset alone (i.e., Z = Zp) or based on both the dataset and the model (i.e., Z Z,, We discuss the normalization in Sec. 3.2.2.
The sample perceptual score Ps.a.y( ( (,,), outlined Fig. 2b, aims to measure the degree to which a model f X + y relies on modality M,, for prediction of sample z E X. To do so we define the

---

sample perceptual score for modality Mm E M as the normalized difference between the accuracy of model which uses all data modalities, and the accuracy of model which doesn't use modality Mn for prediction, i.e, as
FFa.y(Mm) ( A, \ {M.)).
Here, M \ (M is operator that removes the influence of modality Mn, from the set of all modalities M. We define this operation formally in Sec. 3.2.1. Note, fx.gg refers to the classical prediction accuracy of dataset sample (x. y) for given trained model f with M the set of all modalities used for prediction, i.e,, A, (MM M)- 11 ((M
Intuitively, the sample perceptual score Py...y( (Mn) high if the accuracy of model that does not consider modality Mn. for prediction is significantly smaller than the accuracy of model which uses all data modalities M. Conversely, if the accuracy doesn't change, irrespective of whether modality Mn is available or not, the model f doesn't I perceive the modality Mn. Note that in cases where the modality Mm irritates the model, the perceptual score can negative.
In the following we discuss how to 'remove' modality Mm from a model f (Sec. 3.2.1) and how to compute the normalization constant (Sec. 3.2.2).
3.2.1 Removing Modality Influence
Removing modality from a trained model is difficult since typical models entangle modalities and compute high-order correlations. Ideally, we need tool that minimizes the impact of one modality while maintaining the other components" functionality. To achieve this, we study permutation-bant approach, i.e., we randomly permute the modality- think permutation is particularly useful for the perceptual score because it is syper-parameter free. This ensures that the perceptual score defined in Eq. (1) is unambiguous.
features among the test set data D. We
Formally, to compute (,, {M,)) don't 't use all modalities from data Ty. Instead, use all modalities but Mm, i.e., 24
Mi.d and append the data z Mm of modality Mm from another data point 2.y. Hereby j is drawn uniformly from (1,.... [D]), ie., from (11, (DI). Taken together we compute the accuracy via
Acc (M \ (Mm) = E,-et,,, s(tam Mm faff y)=y]-
(3)
In the following we discuss how to compute the normalization Z employed in Eq. (1).
3.2.2 Normalization
Normalization enables comparability. For this, normalization aims for consistency of the perceptual score between models designed for the same task and between models designed for different tasks. Two types of normalization are useful to consider: 1) task-normalization which enables a more meaningful comparison of the perceptual score across different tasks; and 2) a nodel--nornaizztionn that enables a meaningful comparison of the perceptual score within the same task. We think models should be analyzed with both kinds of normalization in mind.
Task fask-lormalizztion: Task normalization is particularly useful if a trivial classifier's 's accuracy is high. Without normalization, the comparison of models designed for different tasks is inconsistent. For instance, if task is relatively easy, the use of permuted data won't 't modify the accuracy much because the task can be addressed with any data modality. This would result in perceptual score close to zero, which isn't 't compelling because, in this case, even a marginal reduction in performance might be significant.
The majority vote classifier always predicts the majority class of the employed training set. We use Acep to refer to the accuracy of the majority vote classifier evaluated on the test set D and compute the normalization factor via
Zp = Accp.
(4)
Intuitively, the normalization factor is the gap between the perfect accuracy (i.e., 1) and the accuracy of the majority vote classifier, i.e. Accp. Hence, normalization by the majority vote accuracy
5

---

amplifies the score difference we compare the obtained gain (classifier accuracy minus permuted classifier accuracy) to the maximally possible gain (one minus majority vote classifier accuracy).
We note that this normalization may result in perceptual score higher than one since the majority vote may be superior to the permuted accuracy. However, this case is unlikely in practice and did not occur in any of our experiments. Notably, this normalization is limited by the fact that the model accuracy is not considered, as we will discuss next.
nodel-normmlizatio::: The performance of a model is an important factor. Suppose for two different classifiers permutation of modality results in the same accuracy that of the majority vote classifier. As a result, the initially stronger model will attain a higher perceptual score. This property can be desirable since one may want the perceptual score to reflect both perception and the ability to address a task. However, in this case the perceptual score does not solely reflect the degree to which a model considers particular modality for decision making. Instead the perceptual score for modality would be conflated with the model's accuracy.
To obtain a score which solely reflects the degree to which model considers particular modality, we normalize by the model's 's accuracy. Formally, we normalize via
ZD.J
(M)).
(5)
4 Evaluation of Perceptual Scores
In the following, assess the perceptual score using popular multi-modal datasets. Specifically, in Sec. 4.1, we study visual question answering (VQA, VQA-CP). We examine video social reasoning (i..., SocialIQ) in Sec. 4.2. In Sec. 4.3, we assess visual dialog models. Our analysis shows that state- of-the-art models exploit biases that haven't 't been documented. We study the bias by investigating samples with low perceptual scores and discover its cause.
We compute the perceptual score based on five permutations per sample. We calculate the perceptual score five times with different permutations and report the mean score along with the standard deviations. We find the expectation to converge quickly and to be stable. For all the models, we used the official implementations. For more details please see our implementation.
Experimental setup:
4.1 Visual Question Answering
The visual question answering task reasons about an image given a question. We use the VQAv2 dataset [20], which contains 443,757 image-question pairs in the train set and 214,354 in the validation set. We also assess the perceptiveness of models trained on "Visual Question Answering: Changing Priors' (VQA-CP) data [21], which was released after several studies suggested that VQAv2 models heavily rely on answer priors. For instance, 'how many' questions are typically answered with '2,' To overcome this shortcoming, VQA-CP suggested a new train-test-splic. As result, the train and test sets have different prior distributions for each question type. The new split consists of 438,183 training samples, and 219 2999228 samples for validation.
Baselines: We use four baselines for VQAv2: 1) BUTD [49], an early competitive approach that used detector-based features pre-trained on VisualGenome [50]; 2) BAN [51], a baseline that uses an effective multi-modal bilinear attention; 3) LMH [41], originally crafted for VQA-CP, this approach removes superficial question patterns; and 4) LXMERT [52] large-scale Transformer-basee model that is pre-trained with large amounts image-text pairs. For VQA-CP, we also analyze: 1) CSS [53] which generates counterfactual training samples by masking critical objects in images or words in questions; 2) RMFE [42] which maximizes the amount of information that each modality contributes to the prediction.
4.1.1 Quantitative Analysis
In Tab. 1, we provide the perceptual score and the accuracy for different basclines and questions. We start by analyzing the perceptual scores of the vision and language modalities. In most cases, models perceive the question better (i.e., Pq > Py). Studying the accuracy, LXMERT has the

---

Table I: Accuracy and perceptual scores on VQAv2 and VQA-CP for different baselines and question types: number (Num), yes/no (Y/N), and other. We report the accuracy (AcoM), the accuracy after removing a modality's s influence (Acc Acc the perceptual score without normalization (Pv, PQ), the perceptual score with task normalization (Pv /Zy/Zp. P2/LD) the perceptual score with model normalization (Pa/ZD,f, Pv Pv/ZD.f) and majority-vote accuracy
(Accp). ). Means and standard deviations are provided.
Image
Question
Model
Q Type AceM
Acc
Pv
Pr/Zo
PulZo.
Pg
Pa/Zo
Palzo.
Acpp
VQAv2
LXMERT
All
68.97
36.46
12.51
47.40 10:0
47.16
1.777
27.41
41.56
60.60
107
60.26
1.33
31.42
LMH
54.33
27.90
38.5 132
0:2
45.94 0.30
57.99
0.3
BAN
AlI
65.67
35.09
30.58
44.59 0.35
46.56
0.39
28.25
37.43
54.57 =0.66
56.99
0.33
31.42
BUTD
All
63.09
34.38
28.71
41.86
45.52
28.78
34.31
50.03
54.33
31.42
LXMERT
Num
52.73
14.55
38.18
47. 14 0.33
72.40
am
13.04
39.69
49.01 0.35
49.00
0:255
19:01
LMH
Num
37.58
15.64
21.94
27.09
58.18
13.75
23,82
29.41
63.40
19:01
BAN
Num
48.62
18.55
30.07
37.13 1:333
61.84 0:2
16.07
32.55
40.19 = 0.24
66.95
0.27
19.01
BUTD
Num
42.46
21.04
21.41
26.44
1:7
50.47
18.39
24.06
29.71
56.29
1.0.
19:01
LXMERT
Other
60.86
16.81
44.05
45.63 0.21
72.37 0:027
3:47
$7.39
59:45
0.08
94.30 0:09
3.47
LMH
Other
54.29
13.72
40,58
42.04 n:1
74.73 +-73:-0..1
4.30
49.99
51.79
92.08
6:000
3.47
BAN
Other
56.99
71.27
0.10
92.68 6o0
BUTD
Other
54.99
14.29
40,70
42, 16 +
73,93
0:22
4.53
50.46
52.27 0.00
91.74
0.07
3.47
LXMERT
85.30
41.02 1.40
24.29 0:032
42.49
0.32
25.16 o.ss
49.49
LMH
Y/N
60,24
50.80
9.43
18.68
+ 0.33
15.66 0.42
50.29
9.94
19.68
0.29
16.50
0.29
49.49
BAN
Y/N
83.03
65,44
17.59
34.82
21.18
64.09
18,93
37.48
22,80
49.49
BUTD
Y/N
80.94
65,42
15.52
30.73
+
19.27 0:33
64.24
16.69
33.05
= 0.30
20.63
0.25
49.49
VQA-CP
CSS
All
$7.89
36,46
21.43
26.43 1:041
37.01 J. 9.777
7.93
49.96
61.61
1031
86.30 1.1.1
18.91
RMFE
All
54.20
29.91
24.29
27.11
47.17
7.65
46.55
51.95
55.91
10.40
LMH
All
52.23
27.14
25.09
28.00 0.3
45.35 024
7.13
45.10
50.33 = 0.14
54.08 0.27
10:40
CSS
Num
51.34
44,50
6.84
7.20
13.32
13:04
38.30
40.34 0.22
74.60 0:1
5.06
RMFE
Num
44.03
37.25
6.77
7.13 + 0.33
18.84 0:33
34.05
9.97
10.51
0.30
26.51 6.33
5.06
LMH
Num
37.40
30,35
7.05
7.43
15.39 0.76
27.48
9.92
10.45 0.35
22.65 a.
5.06
CSS
Other
46.48
36.36
37.57 1.015
78,22 0:15
4.12
43.77
L. 0..66
91.13 1.144
3:23
RMFE
Other
45.97
10.08
35.89
37.09
77.94 0.15
4.15
41,82
43.22
u.11
91.14
3.23
LMH
Other
46.10
35,93
37.13
+..15
0.19
4.08
42,02
43.42
0.11
90.96
6:23
CSS
Yes/No
83.11
82,52
0.59
0.71 1 nat
3.16 Loo
43.84
39.27
100.0
0.0
47,25 1.m
64.46
RMFE
Yes/No
74.47
62.49
11,98
18.58 2.33
17.99
15.16
23:51
21.68 =::31
35.52
LMH
Yes/No
73.75
60.49
13.27
20.58 1.
16.08 1
$7.76
16.00
24.81 0.70
20.35 am
35.52
highest accuracy (68.97%). In contrast, LMH, which reduces the reliance on question priors, achieves the lowest accuracy (54.33%) However, the model- normalzze perceptual score for the visual modality (Pv /DD,)) suggests: LMH perceives image data similarly to other models. Studying the task normalized perceptual score for the visual modality (Pv/Zn suggests: despite the high accuracy on 'Y/N' questions, their image and question perceptual scores are very low, iee. models mostly ignore the visual data and rely on priors.
Next, we show metrics for VQA-CP, variant designed to reduce bias caused by answer priors. We compare different models on all the data by analyzing the model-normalized perceptual score for both visual and question modalities (Pv (Pv ZZD. and Pa. Pg/Zp,y). Interestingly, the Mtate-of-the-an model, CSS, has the lowest image perceptual score (37.01%) %) and the highest question perceptual score (86.30%). suggesting that the question modality may serve shortcut to answer without perceiving the image. Further, by analyzing the different question types via the task-normalized score (Pv DD)) note that CSS has a perceptual score for the image modality of only 3.6% for 'Y/N' questions. One possible explanation: CSS generates new samples, which in turn alter priors perceived during training. We further investigate this hypothesis next.
4.1.2 Bias Analysis for CSS
The Counterfactual Samples Synthesizing (CSS) model produces counterfactual training samples by masking either critical objects in images or words questions, and by assigning different ground- truth answers. Our perceptiveness study above shows that the CSS model has significantly lower perceptual score for the visual modality, despite being state-of-the-ar on VQA-CP with substantial accuracy gap of 6.5% over LMH. Why?
7

---

Table 2: Proportion of yes/no ratios for different kinds of questions. The initial token categorizes questions. We report proportion in the train set. the test set, and the model prediction. * Train'
indicates the number of samples in the train set, # Test' the number of samples the test set.
Token
Model
Predicted Yes
Predicted No
Test Yes
Test No
Train Yes
Train No
# Test
# Train
CSS
1.0
0.0
1.0
0.0
0.5
0.5
1414
2686
'has"
LMH
0.47
0.53
1.0
0.0
0.0
1.0
1414
1343
CSS
0.94
0.06
0.72
0.28
0.5
0.5
3110
2546
'can'
LMH
0.38
0.62
0.72
0.28
0.0
1.0
3110
1273
CSS
0.01
0.99
0.0
1.0
0.5
0.5
324
654
'is
LMH
0.37
0.63
0.0
1.0
1.0
0.0
324
327
'do'
0.99
0.01
1.0
0.0
0.5
1388
11302
LMH
0.42
1.0
0.0
0.44
0.56
1388
5651
The first point to note: CSS generates new samples which may shift prior distributions leveraged during training. Recalling that VQA-CP was introduced to prevent benefiting from answer priors in VQAv2 reveals potential reason for the improvements of CSS.
Use of the sample perceptual score permits more in-depth analysis. In Tab. 2, we identify popular question start tokens with low sample perceptual scores for the visual modality ( ('do,' 'has,' 'can,' 'is a'). We further examine their prediction accuracy using both CSS and LMH. We find that the proportion of 'yes' and 'no' answers between the train and the test set differ: the yes answer is correct for 88% of the questions in the test set starting with 'has,' while the yes answer is never correct for the corresponding training set questions. For CSS, counterfactual samples, however, produce equal proportions. As result, CSS seemingly alleviates the prior inconsistency and adjusts the majority of its predictions to 'yes," which more closely resembles the test set.
Use of the perceptual scores hence permits to hypothesize: improvements in CSS can be attributed to shifted prior distribution instead of complex counterfactual data-manin manipulation. To test this hypothesis we train the LMH model using samples obtained from CSS training. Importantly, we did modify the image or the question. Without even any changes to the imput, we obtain an accuracy of 57.54%, only 0.3% lower than CSS and within standard deviation.
4.2 Video Social Reasoning
SociallQ [28] proposes an unconstrained benchmark, specifically designed to understand social situations. More concretely, given an input tuple of video, question, and an answer, the task is to predict whether the answer is correct not. The videos were collected from YouTube and annotated by students. The dataset is split into 37,191 training samples, and 5,320 validation set samples.
Baseline: We were able to reproduce the SociallQ baseline [28] and achieve an accuracy of 64.84% using all feature modalities during training. In fact, we achieved an accuracy of 67.38% by changing the baseline's ' model to FGA [54] and by using GloVe [55] instead of BERT features [56].
4.2.1 Quantitative Analysis
In Tab. we show scores for different metrics. The task-normalized perceptual scores (Par for different modalities M (first column) of FGA and the baseline reveals heavy reliance on the answer modality and low dependence on the video modality (i.e., 15.24 vs. 3.25). To verify that answer is indeed the only important modality, we train classifier using only the answer modality and observe: we can surpass the original paper's bascline by 4% when only using answer features, achieving 68.65% accuracy. Again, the perceptual score helped us understand shortcomings of model. Upon analyzing the bias source, we observe sentiment bias, which we discuss next.
4.2.2 Bias Analysis for SocialIQ
Assessing samples with low sample perceptual scores can reveal biases. Our study suggests that the labels correlate strongly with the sentiment. In Fig. 3, we marked with a red border two videos with a low sample perceptual score for the video modality. Studying those and similar samples, we find:
8

---

Table 3: Accuracy and perceptual scores on SociallQ. For different modalities (first column), we report the accuracy M), the accuracy after removing the modality's s influence (Acc,)) (M) (MM) the perceptual score without normalization (PM), the perceptual score with task normalization (PM /ZZ),, the perceptual score with model normalization (PM / D.. f). and the majority vott accuracy (Acep).
Modality M
Model
AceM
ACCM)((M)
PM
Pa/Zo
PM/ED.
Accp
Baseline
64.84
56.73
8.11
18,92 0.13
12.51 0.03
57.14
Answer
FGA
67.38
57.11
10.27
23.96 +: 021
15.24 0.10
57.14
Baseline
64.84
64.32
0,52
1.21 0.02
0.80 +
57.14
Question
FGA
67.38
65.19
2.19
5.11 1.1.1
3.25 1 0.08
57.14
Baseline
64.84
63.79
1.05
2.45 0.05
1.62 + 0.03
57.14
Video
FGA
67.38
64.42
2.96
6.91 0.16
4.39 = 0.15
57.14
NLTK-Sentingg
66.70
56. 14
11.18
26.08 + 0.19
16.36 0.09
57.14
Answer-Only
68.65
57.29
11.39
26.57 1: 0.21
16.59 0.06
57.14
Quastion: Why doeent the older womeon admil the okay?
How
Question: How do man woman feel about each
other?
Answer: SSee wents remain strong
Anewer: bay feets socthed.
Ansoer: They feel insomfortttle anound each other
Ground Truth: Troe
Ground Truth: True
Ground Truth: Falew
Negstive
Figure 3: SociallQ data samples. On the left, we show a sample with a high perceptual score towards video data. Neither positive nor negative sentiment is evident in this sample. Hence, the video is required for prediction. We illustrate two samples (marked with red border) that received a low perceptual score. There is sentiment-based correlation between the label and the answer these samples. For simplicity, we highlight with red color words that exhibit sentiment.
1) when the answer is True, the answer has positive sentiment; 2) in contrast, when the answer is False. the answer contains words with a negative connotation, e.g., "uncomfortable."
We hypothesize: successful prediction of the answer by just looking at the answer modality for SociallQ data is due sentiment-biaa -iiseed annotations. To validate this hypothesis, the-shelf sentiment classifier from the NLTK package [57]. When applied to SociallQ without any training, obtain remarkable answer prediction accuracy of 66. 7%. This matches our reported result of 68.65% quite reasonably and outperforms the SociallQ baseline [28].
off-
4.3
Visual Dialog
The visual dialog task encourages models ask and answer questions about visual input. Notably, each iialog-interaction employs many modalities (e.g., image, question, caption, dialog-histors We show our results the VisDial v1.0 dataset, where 123,287 images used for training, 2,000 images for validation, and 8,000 images for testing [58]. Each image is associated with ten questions, and each question has 100 corresponding answer candidates.
Instead of accuracy Acc (M). here, we use ranking-bassee metrics as the Visual Dialog dataset primarily uses two metrics: MRR and NDCG, In short, the MRR metric examines the rank of single ground-truth response (i.e., sparse annotations), while the NDCG metric measures the cumulative gain in the case of multiple correct answers (i.e., dense annotations). We computed the majority ranking based on answer frequency in the train set. See appendix for more.
We use two baselines for VisDial v1.0: 1) FGA [54], an attention unit inspired from graphical models that infers an attention map for each modality; and 2) LS [59], which pre-trains on related vision-language datasets, e.g., Conceptual Captions and Visual Question Answering [1, 60]. We also report L.S (CE), which finetunes on the dense annotations, the expense of MRR performance.
Baselines:

---

Table 4: Perceptual scores on VisDial v1.0. We show that the perceptual score can be computed using metrics other than accuracy (i.e., MRR and NDCG).
Medallty M
Model
MiRK
(R
P3o
Plr 120
MRR
NDCGs911M)
Pla/2e
Ptx/2fo. r
NDCGa
(CB)
2.010
20.35
54.89
73.56
11514
2.000
Questox
21.04
3:000
55.13
5.622
PGA
50.03
30.74
25.25
25.65
(CFC
$2.21
34.94
15.32
35.45
1000
38.08 4/00
52.32
75.36
$7.45
17.39
23.06 2.13
33.64 000
25
Image
1.
69.00
52.05
16.94
34.99
600
24.55
4.000
32.22
64.50
$1.50
13.39
18.02
20.63 000
25
PGA
00.14
52.15
14.00
20.05
21.16
9:000
32.22
50,00
46.73
9.27
0.27
2.00
16,55 2:00
25.08
(CE)
52.21
52.18
0.03
0.00
2
0.06 0: 3.90
22.22
75.24
75.39
0.05
0.07 1
0.17 2:001
25.68
Capáaan
69.00
0.00
0.80
3/0
0 3:90
64.89
0.00
0.00 4-00
4:000
PGA
66.14
1.21
1.79 302
3:2
55.32
0.58
0.92
Table 5: Perceptual scores for audio visual crowd counting on high and low resolution settings.
Image qualitt
1- MAPESt
1 MAPEx1.
PA
Pa
Ze
Pa Zp.
1
MAPE
Pv.
Pv V.p
Pv Zo..
1- MAPE
High
27.71
34.55
83.21
44.46
Low
72.51
66.58
9.93
23.91
13.69
45.07
27.4
66.88
37.84
58.48
4.3.1
Quantitative Analysis
In Tab. 4, we show the perceptual scores for different modalities (i.e., M), and for two metrics: MRR and NDCG. We find: 1) using the MRR metric, the model-normalized perceptual score for the question modality of FGA is relatively high compared to LS (i.e., 51.26% vs. 30.49%). However, when we employ the NDCG metric, the LS model perceives the question better (i.e.. 63.14% vs. 45.11% 2) analyzing the only model optimized for NDCG, i.., LS (CE), reveals: When analyzed using the NDCG metric, the model-normalized perceptual score for both image and question is significantly higher than for MRR. Interestingly, the LS (CH) model-normalized perceptual score for the image modality is on par with the other models when we use the MRR metric. However, the question perceptual score is low, which suggests the reason for the low MRR performance may be related to the low utilization of the question. Finally, we note that all models, on both metrics, seem to ignore the caption, which suggests caption information is redundant.
4.4 Audio-visual Crowd Counting
We further investigate a model for audio-visual crowd counting task. Given an image and an utterance, the task is to count the number of people in a scene. We use the auDiovISual Crowd cOunting (DISCO) dataset, consisting of 1,935 images and the corresponding audio clips, and 170,270 annotated instances. Hu et al. [61] hypothesize that in extreme cases (i.e., low-quality images), the scene audio can improve the prediction. As expected, the perceptual score is able quantify the degree to which each modality is perceived. In Tab. 5 we compare the low-quality and high- high-quality images by employing task normalization. It reveals that in high-quality image settings, the audio modality is mostly ignored (the normaoora audio perceptual score is 9 9.22%). In contrast, the audio modality perceptiveness for low-quality image settings is significant (the normaioral audio perceptual score is 23.91 23.91%).
5 Conclusion
We introduced the perceptual score a multi-modal classifier towards data modality. The perceptual score assesses a classifier's perceptiveness of modality and reveals exciting insights if analyzed carefully. We hope that this is demonstrated by our studies which reveal that 1) shifted prior distributions seem to help CSS achieve Mate-of-the-aue results, 2) SociallQ data exhibits sentiment bias, 3) Visual Dialog caption information appears less informative, and 4) Audio perceptiveness for vidiol crowd counting increases when low-quality images were present. We hope that researchers working on multi-modal models see the use of the perceptual score and will start to report the perceptual score in addition to classical accuracy metrics.
Limitations: We propose perceptual scores, a novel metric that conveys information about multi- modal classifiers. The two limitations we can see: 1) small computational overhead; 2) false sense of security. Perceptual scores don't 'tn't alleviate the need to carefully study results. However we think the socictal and scientific benefits of reporting this novel metric outweigh the concerns.
10

---

Acknowledgements: This work is supported in part by NSF Grant #1718221, 2008387, 2045586. 2106825. MRI #1725729, and NIFA 2020-67021-32799. We also acknowledge support from Samsung, Amazon, and Cisco Systems Inc. (Gift Award CG 1377144/thanks for access to Arcetri),
References
[1] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitiick, and Devi Parikh. VQA: Visual Question Answering. In ICCV, 2015. 1, 9
[2] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image database. CVPR, 2009.
[3] Lin Tsung- Yi, Maire Michael, Belongie Serge, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014.
[4] Vinod Nair and Geoffrey E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
[5] Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. ICML, 2015.
[6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
[7] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017. 1
[8] Anqi Liu and Brian Ziebart. Robust classification under sample selection bias. NIPS, 2014. 1
[9] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. JMLR, 2014.
[10] Pierre Stock and Moustapha Cisse. Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases. In ECCV, 2018.
[11] Idan Schwartz, Alexander G. Schwing, and Tamir Hazan. simple baseline for audio-visus scene-aware dialog. In CVPR, 2019. 3.
[12] Yann Lecun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. In Proceedings of the IEEE, 1998. 2
[13] Sameer A. Nene, Shree K. Nayar, and Hiroshi Murase. Columbia object image library (coil-20).. Technical report, Columbia, 1996.
[14] Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: sengguage-independeen named entity recognition. In NAACL, 2003. 2
[15] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012. 2
[16] Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In CVPR, 2015.
[17] Mor Geva, Y. Goldberg, and Jonathan Berant, Are we modeling the task or the annotator? an investigation of annotator bias in natural language understanding datasets. In EMNLP, 2019.
[18] Allan Jabri, Armand Joulin, and Laurens van der Maaten. Revisiting visual question answering baselines. In ECCV, 2016. 3
[19] Daniel Rosenberg, Itai Gat, Amir Feder, and Roi Reichart. Are VQA systems rad? measuring robustness to augmented data with focused interventions, ACL, 2021. 3

---

[20] Yash Goyal, Tejas Khot, Douglas Summers-Ster Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In CVPR, 2017. 3, 6
[21] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don't just assume; look and answer: Overcoming priors for visual question answering. In CVPR, 2018. 3, 6
[22] Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K. Marks, Chiori Hori, Peter Anderson, Stefan Lee, and Devi Parikh, Audio-visual scene-aware dialog. In CVPR, 2019. 3
[23] Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In EMNLP, 2015. 3
[24] Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation artifacts in natural language inference data. In NAACL (Short Papers), 2018. 3
[25] Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In NAACL, 2016.
[26] Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In CoNLL, 2017. 3
[27] Damien Teney, Kushal Kafle, Robik Shrestha, Ehsan Abbasnejad, Christopher Kanan, and Anton van den Hengel. On the value of out-of-distribution testing: An example of goodhart's law. NeurlPS, 2020.
[28] Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency, Social- iq: A question answering benchmark for artificial social intelligence. In CVPR, 2019. 3,
[29] Mohsan S. Alvi, Andrew Zisserman, and Christoffer Nelläker. Turning blind eye: Explicit removal of biases and variation from deep neural network embeddings. In ECCV WCCCCCC (Workshop), 2018.
[30] Shikha Bordia and Samuel R. Bowman, Identifying and reducing gender bias in word-level language models. In NAACL (Worksho)), 2019.
[31] Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. Racial bias in hate speech and abusive language detection datasets. In ACL, 2019.
[32] Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In ECCV, 2018.
[33] Julio Cezar Silveira Jacques, Cagri Ozcinar, Marina Marjanovic. Xavier Baró, Gholamreza Anbarjafari, and Sergio Escalera. On the effect age perception biases for real age regression. FG 2019, 2019.
[34] Kimmo Kärkkäinen and Jungseock Joo. Fairface: Face attribute dataset for balanced race, gender, and age, 2019. URL https:://axiisi httppp:://aaav.ooog/oo////////// /900. pdf...f
[35] Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning not to learn: Training deep neural networks with biased data. In CVPR, 2018.
[36] Tianlu Wang. Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In ICCV, 2019.
[37] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In NAACL, 2018.
12

---

[38] Ran Zmigrod, Sabrina J. Mielke, Hanna Wallach, and Ryan Cotterell. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. In ACL, 2019. 3
[39] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In EMNLP, 2017.
[40] Remi Cadene, Corentin Dancette, Matthieu Cord, Devi Parikh, et al. Rubi: Reducing unimodal biases for visual question answering. In NeurIPS, 2019.
[41] Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. Don't 't take the easy way out: Ensemble based methods for avoiding known dataset biases. In EMNLP, 2019. 6,6
[42] Itai Gat, Idan Schwartz, Alexander Schwing, and Tamir Hazan. Removing bias mudal classifiers: Regularization by maximizing functional entropies. In NeurlPS, 2020. 6
[43] Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee. Overcoming language priors in
visual question answering with adversarial regularization. In NeurlPS, 2018. 3
[44] Geoffrey Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 2002. 3
[45] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you?: Explaining
the predictions of any classifier. In SIGKDD, 2016. 3
[46] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. In /CML, 2017. 3
[47] Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In NeurlPS, 2017. 3
[48] Jack Hessel and Lillian Lee. Does my multimodal model learn cross-modal interactions? it's harder to tell than you might think! In EMNLP, 2020. 3
[49] Peter Anderson, Xiaodong He, Chris Buchler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In CVPR, 2018. 6
[50] Ranjay Krishma, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanic Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, Michael Bernstein, and Li Fei-Fei. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 2017. 6
[51] Jin-Hwa Kim, Jachyun Jun, and Byoung- Tak Zhang. Bilinear attention networks. In NeurIPS,
2018. 6
[52] Hao Tan and Mohit Bansal. Lxmert: Learning cross-modality encoder representations from transformers. EMNLP, 2019, 6
[53] Long Chen, Xin Yan, Jun Xiao, Hanwang Zhang, Shiliang Pu, and Yueting Zhuang. Counter- factual samples synthesizing for robust visual question answering. In CVPR, 2020. 6
[54] Idan Schwartz, Seunghak Yu, Tamir Hazan, and Alexander G. Schwing. Factor graph attention. In CVPR, 2019. 8,9.
[55] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In EMNLP. 2014. 8
Ming-Wei [56] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. NAACL, 2019.
[57] Edward Loper and Steven Bird. Nltk: The natural language toolkit. In ACL, 2004. 9
13

---

[58] Abhishek Das. Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, José MF Moura, Devi Parikh, and Dhruv Batra. Visual dialog. In CVPR, 2017. 9
[59] Vishvak Murahari, Dhruv Batra, Devi Parikh, and Abhishek Das. Large-scale pretraining for visual dialog: A simple state-of-the-art baseline. ECCV, 2020. 9
[60] Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In ACL,, 2018. 9
[61] Hu Di, Lichao Mou, Qingzhong Wang, Junyu Gao, Yuansheng Hua, Dejing Dou, and Xiao Zhu. Ambient sound helps: Audiovisual crowd counting in extreme conditions. In CVPR, 2020. 10
14


Reviews:
[{"review_id": "yypPhJh9HBQ", "reviewer": "Reviewer_SjCY", "summary": "This paper aims to quantify the degree to which a model relies on the different subsets of the input features (i.e., modalities), and introduces the perceptual score. In experimental, they find that state-of-the-art multi-modal models for visual question-answering or visual dialog tend to perceive the visual data less than their predecessors.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "1.\tThe topic to explore the importance of different modalities is interesting and the reviewer thinks that it will bring more interpretability for multi-modal learning\n2.\tThe proposed method is direct and easy to follow. And the experiments find that multi-modal models tend to ignore some modalities while taking shortcuts.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Weakness:\n1.\tThis manuscript is more like an experimental discovery paper, and the proposed method is similar to the traditional removal method, i.e., traverse all the modal feature subsets and calculate the perceptual score, removing the last subset. The reviewer believes that the contribution of the manuscript still has room for improvement.\n2.\tThe contribution of different modalities of different instances may be different, e.g., we have modalities A and B, some instances with good performance of modality A  which belongs to the strong modality, whereas some instances with good performance modality B which belongs to the strong modality. Equation 3 directly removes the modal subset of all instances. How to deal with the problem mentioned above.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "wDUzn1CU_DF", "reviewer": "Reviewer_NSvs", "summary": "This paper pays attention to analyzing the perceptiveness of multi-modal classifiers, proposing a perceptual score, which is very helpful for the community. Using the proposed perceptual score, the authors analyze a wide range of models on different datasets, finding that multi-modal models tend to ignore some modalities while taking shortcuts. In terms of bias, this paper finds that SocialIQ is biased by sentiment. The above interesting findings are helpful for researchers to propose new models and develop new datasets.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The proposed perceptual score is simple and can be easily computed, which could be a useful metric to measure the perceptiveness of multi-modal classifiers. The perceptual score is novel to me. The analysis is comprehensive, including many existing models on several multi-modal datasets. And the findings are inspiring, which is helpful for the community.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "1. some models that use more information should be analyzed, such as Oscar [1].\n\n2. this paper only analysis the vision-language datasets, but there are some datasets composed of other modalities (vision, audio and language).\n\n[1]. X. Li et al.. Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. ECCV, 2020.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": ["I don’t know"], "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "kdnWknaIdVe", "reviewer": "Reviewer_mYsn", "summary": "This paper proposes a measure for evaluating multimodal models on how much they take into account the various modalities in making their predictions. The proposed \"perceptual score\" evaluates a model's performance on each example when one of its modalities is randomly sampled from other items in the dataset (an expectation of performance when paired with all possible versions of the modality being tested). The general idea is that if the performance when paired with a random sample is similar to the performance when paired with the ground truth modality, then the model is likely not taking into account that modality when making its prediction.\n\nThe paper evaluates the metric on several vision and language tasks (VQA, VQA-CP, SocialIQ, and VisDial) with several interesting findings (e.g., that one can achieve relatively high performance on SocialIQ without even training a model and only considering one of the modalities).", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: This is my main concern with the paper and why I am giving a score of 5 in my initial review. Hessel and Lee (EMNLP 2020, https://aclanthology.org/2020.emnlp-main.62.pdf) propose a very similar measure for the same problem studied in this paper, called EMAP. The intuition of the two measures is very similar: measuring the performance on a random sample of modalities. I didn't have time to compare both papers exactly, but I would encourage the authors of this paper to not only cite Hessel and Lee 2020, but include space to compare the measures (since the EMNLP paper has been published for a while now, I would expect the authors of this paper to know about it, and am surprised it was not cited). At a surface level, it does seem the measures are slightly different; for example, they compute a single metric for a dataset while this paper proposes to provide metrics for each modality independently (although I am not sure if this is just \\hat{f}_T and \\hat{f}_V in Eq 3 of Hessel and Lee 2020). The paper in submission also includes discussion of some normalization used to compare models (and tasks?) directly, and additional analysis on several datasets. \n\nI would expect a final version of this paper to include a discussion of how the method proposed in this paper compares with the measure introduced by Hessel and Lee 2020. Even if the measures are equivalent, this paper does offer additional analysis and variants of the measure that may be interesting standing on their own, although it may require significant reframing of the paper.\n\nQuality: I would have liked a more theoretical intuition of the normalization methods (3.2.2). I was also somewhat confused about when you would use the two different kinds of normalization or how to combine them.\n\nThe experiments and analysis are quite thorough, and the finding on SocialIQ is very interesting.\n\nNit: in lines 160--161, \"If the accuracy doesn't change, irrespective of whether modality M_m is available or not, the model f *doesn't* perceive the modality M_m\" while this is very *likely* the case, I don't think we can conclude that it doesn't perceive the modality at all; after all the metric is an estimate only (expectation over items in the dataset).\n\nClarity: The paper is easy to follow and understand for the most part. Being more clear about what \"bias\" means here could be useful, since this term is ambiguous. \n\nSmaller things:\n- Table 1, may be useful to have a reminder of whether a high or low perceptual score is good.\n- Table 3: since Acc_D is not dependent on the models maybe don't have different numbers in each model row, rather just have its own combined row. \n\nSignificance: A measure like this could be useful for researchers working on multimodal problems. The techniques used in this paper to analyze SocialIQ and VQA-CP provide a good framework for people to interpret and deeply analyze the perceptual scores and their implications on a model and dataset. I worry though that there is significant overlap with EMAP that will cause confusion between the two measures, so would appreciate the authors clearing this up.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Although brief, I appreciated the discussion of a \"false sense of security\" these measures will provide. I have similar concerns that these kinds of metrics will be taken as fact (e.g., that high perceptual scores mean a model/dataset don't have any (inductive) bias). ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "Qz8LXkWgsZ4", "reviewer": "Reviewer_bqxx", "summary": "The paper proposes a score to understand biases in datasets and models by trying to assess the reliance of a model on different modalities. Specifically, this perceptual score is defined as the difference between the model’s accuracy with all modalities and with one modality removed. The proposed method is  used to bring insights into significance of text and visual modalities for different question answering systems. ", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper proposes a method to understand deep neural network models. In particular, multimodal systems are often hard to interpret and the significance of different modalities in solving the task at hand is often not well understood. This is clearly an important problem and in that the paper is clearly well motivated in attempting to do that. \nThe paper is clearly written and mostly easy to follow. \n\nThe current definition of perceptual score and its role in understanding the significance of different modalities as well as biases in models is not very convincing. \n\n1. The perceptual score is showing how the overall accuracy is changing. However, its unclear what this means at sample level. How do we interpret prediction label flip (from correct to incorrect as well as incorrect to correct)  at sample level when “removing” a modality. \n\n\n2. The modalities are not really removed and I agree that it would be difficult to do for most systems. However, randomly swapping the modality from another data point in the test set is not convincing as well. How do we understand this randomness or stochastic nature of the perceptual score ? Moreover, using the modality input from another data point biases it towards the dataset again. Wouldn’t it be better to just use random “noise” to remove a modality -- say for example some sort of gaussian-noise image input to “remove”. Can this work ? One can also use random image or text from a different dataset. \n\n\n3. It is unclear how  including Z_D as a normalizing factor helps include the difficulty of a task in the perceptual. Can the authors clarify how  normalization by this factor is really helpful in grounding the perceptual score. If the training data consists of almost even distributed classes then \\hat{Acc}_D is basically the 1/#classes which can be large or small depending on the number of classes. \n\n\n4. The empirical analysis in the paper has focused on using the metric to draw insights into different tasks and systems. However, some empirical analysis on why the metric itself makes sense is necessary ? How do we understand that score of this form is a reliable way to understand datasets and models ?  The paper does not compare or even relate the proposed perceptual score directly to any prior methods which makes it even harder to understand its significance. \n\n\n5. I am not able to fully understand how the perceptual score aids the bias analysis of CSS. From Table 2, it does look like that CSS is achieving better performance by balancing class priors which are changing from train to test.  But what is the role of perceptual score here ? The numbers in Table 2 and conclusions from it can be drawn without the perceptual score.   Do we need the perceptual score to do this analysis ? \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The paper proposes methods to help understand deep multi-modal models. The authors themselves have outlined two limitations in their conclusions, although they have not been discussed in detail. \n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: Understanding how sensitive multi-modal models are to their various inputs is an important topic and this submission proposes a straight-forward to estimate these perceptual sensitivities. The proposed method is demonstrated in a couple of domains and the author response extended to an additional setting. Reviewers unanimously recommend acceptance and the AC concurs. Authors are strongly encouraged to include the additional audio setting in the camera ready and to clarify the comparison to EMAP.
Decision: Accept (Poster)