Year: 2019

Abstract:
In order to make Visual Question Answering (VQA) explainable, previous studies not only visualize the attended region of a VQA model but also generate textual explanations for its answers. However, when the model’s answer is ‘no,’ existing methods have difficulty in revealing detailed arguments that lead to that answer. In addition, previous methods are insufficient to provide logical bases, when the question requires common sense to answer. In this paper, we propose a novel textual explanation method to overcome the aforementioned limitations. First, we extract keywords that are essential to infer an answer from a question. Second, for a pre-trained explanation generator, we utilize a novel Variable-Constrained Beam Search (VCBS) algorithm to generate phrases that best describes the relationship between keywords in images. Then, we complete an explanation by feeding the phrase to the generator. Furthermore, if the answer to the question is “yes” or “no,” we apply Natural Langauge Inference (NLI) to identify whether contents of the question can be inferred from the explanation using common sense. Our user study, conducted in Amazon Mechanical Turk (MTurk), shows that our proposed method generates more reliable explanations compared to the previous methods. Moreover, by modifying the VQA model’s answer through the output of the NLI model, we show that VQA performance increases by 1.1% from the original model.

heading: 1 INTRODUCTION
text: Recently, the interpretability and reliability of Artificial Intelligence (AI) are being seriously discussed (Kim et al., 2018b), since the opacity of its internal structure makes it hard to figure out the exact reason when it malfunctions (Gunning, 2017; Kurakin et al., 2016). Previous works show that decision process of conventional machine learning models (e.g., decision tree) can be modified to be transparent while maintaining their original performance (Si & Zhu, 2013; Letham et al., 2015). For deep learning models, Ribeiro et al. (2016) suggest a method for reasoning their decisions by visualizing critical parts of the input for the image classification. However, it is harder to infer the decision process for more complicated tasks such as VQA.
VQA is one of the most active research fields in AI with strong expectation for real applications. It can be used for either assisting radiologists (Letham et al., 2015) or help visually impaired people to understand their environment more flexibly (Antol et al., 2015). Therefore, state-of-the-art VQA models have been continually proposed, but their internal processes are yet opaque. Displaying attended region might be used for indirect reasoning. However, it has limitation on describing what the model perceives in detail. The most spotlighted recent research for endowing interpretability to VQA model is making the model to produce an answer and its explanation at the same time (Park et al., 2018; Li et al., 2018; Hudson & Manning, 2019). To make it possible, they provide a ground truth explanation dataset for each image, question and answer pairs, and fine-tune the model to generate explanations in an end-to-end process.
Previous explanation models can successfully describe relationships between objects in the image. However, when the answer to the question is “no,” the previous methods have difficulty on providing logical bases. Also, even though there are questions that require common sense, existing models
do not utilize it: previous models memorize explanations appropriate to each situation in training datasets (Park et al., 2018; Zellers et al., 2019).
In this paper, we propose a novel method that generates explanations for a VQA model’s answer. First, the proposed method selects keywords that play an essential role in inferring the answer to the question. We extract these keywords in the same way as Wang et al. (2019). Next, given a pre-trained explanation generator, we use the proposed Variable-Constrained Beam Search (VCBS) algorithm to generate a phrase that best describes the relationship between the keywords by inferring the image (See Fig. 1). The phrase is fed to the generator to complete the remaining part of the explanation. Finally, we use Natural Language Inference (NLI) (Wang et al., 2019; Talman et al., 2019) to check whether the contents of the question can be inferred from the explanation using common sense. For the overview of our method, please refer to Fig. 4.
Our main contributions are:
• We propose the novel method to generate textual explanations for the VQA model’s answer. The proposed method not only produces more appropriate explanations (Kim et al., 2018b) but can also be used as a post-processing of the VQA model to boost the performance. We validate our method on one hundred randomly sampled data from VQA 2.0 test-std split, through a user study conducted in MTurk. By modifying the predicted answer using the generated explanation, we also compare the prediction accuracy of the method on complete VQA 2.0 test-std split. Our proposed method has gained a MOS score higher than previous methods, and get a VQA performance improvement of about 1.1%.
• We propose the Variable-Constrained Beam Search (VCBS) algorithm that helps to identify the visual relationship between objects and attributes of interest in the image. Unlike the previous Constrained Beam Search (CBS) algorithm (Sipser et al., 2006), the VCBS has the variable accepting states. The variable accepting states help to satisfy the given constraint as much as possible but allow the final beam to exclude some constraints if necessary.

heading: 2 RELATED WORKS
text: We first summarize recent research in VQA. Then we review the explanatory model for VQA and discuss the distinguishing characteristics of our work.
Visual Question Answering: The VQA challenge was first launched in 2016, and representative studies at that time (Fukui et al., 2016; Lu et al., 2016; Kim et al., 2017) first extract image features
using residual networks (He et al., 2016) and established joint learning through question features and attention obtained from gated recurrent units (GRUs). In the VQA 2017 challenge, Anderson et al. (2018b) proposed a method that exploits Faster-RCNN (Ren et al., 2015) to extract image features for the attended object candidates. Following its scheme, most of the subsequent VQA studies also used Faster-RCNN. In the VQA 2018 challenge, Kim et al. (2018a) presented bi-linear attention so that both question features and visual features were considered together when generating attention. They have shown that bilinear attention improves the residuality of the attention presented in (Kim et al., 2016) successfully.
Although various attention-based VQA models have shown excellent performance, the models heavily rely on linguistic bias rather than visual information in most cases (Ramakrishnan et al., 2018). For this reason, recent studies are interested in identifying relationships among objects in images and using them to predict answers (Hudson & Manning, 2019).
Explanatory model for VQA: Visualizing attention (Lu et al., 2016) or utilizing Grad-CAM (Selvaraju et al., 2017) is one of the most widespread approach to understand the decision process of the VQA model. However, as mentioned in Li et al. (2018), we need detailed textual explanations rather than merely visualizing these areas. For this reason, recent research not only released ground truth explanation datasets (VQA-X (Park et al., 2018) and VQA-E (Li et al., 2018)) but also proposed models that generates an answer and its textual explanation at the same time. While VQA-X is made up of more elaborate and detailed explanations, VQA-E is composed of more extensive training and validation data. According to the results of the Li et al. (2018), fine-tuning through the VQA-E dataset increases the prediction accuracies of VQA models noticeably.
While both of the models are good at justifying the answer for the given question and image, they are poor at providing introspective explanations of the VQA model. First, the PJ-X model proposed by Park et al. (2018) consists of the sequence of a VQA model and an explanatory model. Since the PJ-X does not refer to the attention generated in the VQA model, the explanation does not reflect the decision process of the target VQA model. On the other hand, the multi-task VQA-E model proposed by Li et al. (2018) directly utilize the attention distribution of pre-trained VQA models. Although the model generates explanations based on visual information about the attended area, it
often produces sentences that are not related to the question. This is because multi-task VQA-E does not directly specify the relationship between the objects related to the question.
Since it is important for VQA task to grasp the relationship between specific objects and attributes in an image, recent studies infer an answer through a scene graph or a modular network formed for the image (Hudson & Manning, 2019; Hu et al., 2018). Although both of them provides explanatory insights into the model, the methods lack a logical basis for the explanation in situations where the answer is “no.” (See Fig. 3).
Natural Language Inference: The goal of NLI is to classify the relationship between two sentences: “premise” and “hypothesis.” If the “hypothesis” can be inferred from the “premise,” we classify the relationship between the two sentences as “entailment.” While most NLI models use LSTM to classify the relationship between the two sentences (Talman et al., 2019), Wang et al. (2019) proposed a method to grasp the relation by using ConceptNet (Speer et al., 2017) (a common sense knowledge graph obtained through crowd-sourcing) as external knowledge. We use state-ofart NLI models (the HBMP (Talman et al., 2019), and the ConSeqNet (Wang et al., 2019)) to identify if the question could be answered using common sense and our explanation (See Fig. 4(b)).

heading: 3 APPROACH
text: For an input image, the Recurrent Neural Network (RNN) in the captioner forms the Variable Order Markov (VOM) model between words (Begleiter et al., 2004). This VOM model can be represented as a directed graph G = (Σ, E), where Σ is the complete vocabulary, and E is probability distribution at each node (See Fig. 1). If the captioner is ideal, the goal of VQA can be regarded as (i) checking whether a path that contains specific nodes can be found (for the question with yes/no answers. See Fig. 1) ; (ii) or completing the path that pass through the specific nodes (for the other type of questions. (Lin et al., 2019)).
Algorithm 1 Variable-Constrained Beam Search decoding 1: procedure VCBS(Θ, b, T, C,A = ( ∑ , S, s0, fSF (p), δ))
2: M ← φ . for Memoization 3: for s ∈ S do 4: Bs ← { } if s = s0 else φ 5: for t← 1 to T do 6: Es ← ⋃ s′∈S{(y′, w)|y′ ∈ Bs ′ , w ∈ ∑ , δ(s′, w) = s}
7: Bs ← arg maxE′⊂ES ,|E′|=b ∑ yE′∈E′ Θ(y
E′) . Beam extension 8: if s ∈ fSF (p) then . for Beams in accepting states 9: M ← arg maxyM∪Bs∈M∪Bs |C ∩ yM∪B
s | . keep beams satisfying more constraints
10: M ← arg maxM ′⊂M,|M ′|=1 ∑ yM′∈M ′ Θ(y M ′) . only keeps the best beam
11: return y ∈M if M 6= φ else φ

heading: 3.1 CONSTRAINTS
text: First, we identify Part-Of-Speech (POS) tags (Màrquez & Rodrı́guez, 1998) of each word in the question. Then we take all detected nouns and verbs (excluding auxiliary verbs) as keywords. If the VQA model’s answer is “yes” or “no,” we add adposition to keywords (See Fig. 1). Otherwise, we use the answer as additional keywords (See Fig. 4(a)).
As in the work of Anderson et al. (2018a), we construct Wordnet synsets (Fellbaum, 2010) containing plural and singular forms for each keyword. Thus, the nodes in Fig. 1 correspond to the synsets of each keyword. We use a set of these synsets as constraints set C in our VCBS algorithm. For simplicity in notation, we assume that C is composed of keywords, not synsets, in the rest of this paper. The set C is further divided into two subsets, strict constraints (Cstrict) and soft constraints (Csoft). Cstrict is a set of keywords that the final beam must satisfy in order to be in accepting state (See Fig. 2). Csoft is a set of keywords that are recommended to satisfy in order for the final beam to be accepted, but not necessarily required. We use the answer as strict constraints only if the answer is not “yes” or “no” (See Fig. 4).

heading: 3.2 VARIABLE-CONSTRAINED BEAM SEARCHING
text: For the RNN in image captioner with parameter θ, a word wt at time step t is predicted given the previous sequence of words {w0, · · · , wt−1} and the image I .
wt = arg max ŵt
pθ (ŵt | wt−1, · · · , w0, I) , (1)
To naturally place the keywords into the sequence, we need to partially specify sequence with Beam Search decoding (Anderson et al., 2018a). During the completion of a beam, whenever the beam satisfies a constraint, we change the state of the beam (See Fig. 4). We denote this state-transition function as δ. The completed beam for a given image may not satisfy all constraints: no clue is found in the image that satisfies the constraint (See Fig. 1.), or the constraint is given as a hyponym (See Fig. 4(a)). Therefore, the satisfaction of constraints for each beam is not compulsory, but recommended (except the Cstrict).
We express this condition as a set of accepting states SF = fSF (p) with a variable p, where fSF (p) : N → 2S , and S is a set of all possible states. The loosening parameter p denotes the number of keywords in C that can be neglected. The overview of our VCBS decoding is depicted in Algoirthm 1. Note that Θ = log pθ(·), b is the beam size, T is the maximum sequence length. Since the completed beams may not stem from the phrase that satisfies the constraint, we keep the best phrase presents in SF through Memoization. Finally, we complete the phrase y ∈ M through the greedy decoding in Equation 1.

heading: 3.3 LINGUISTIC INFERENCE
text: Some of the questions in VQA require inferences based on background knowledge in addition to visual information. Most of these types of questions require complex inference processes (Lin et al., 2019) but can be reasoned relatively quickly when the answer is yes/no : by checking whether the contents of the question can be inferred from visual information. Consequently, we first convert the question to a statement by moving the auxiliary verb to behind the subjective noun (e.g., Are dogs on the grass? ⇒ Dogs are on the grass). Using this statement as “hypothesis” and the complete sentence from y as “premise,” we checked whether they are in the “entailment” relationship (See Fig. 6(c)).
For the knowledge extension, we extract concepts from “premise” and “hypothesis,” and identified their relationship in ConcepNet Knowledge Graph (ConceptNet KG). As in Wang et al. (2019), we use nouns and verbs (excluding auxiliary verbs) as concepts for “premise.” However, for “hypothesis,” we only use elements of remaining constraints C ∩ yc as concepts (See Fig. 4(b)). Finally, we use Concepts Only (Wang et al., 2019) and One-Hop Direct subgraphs with knowledge graph embedding (Wang et al., 2014) for NLI. The One-Hop Direct subgraph only connects concepts from “premise” to “hypothesis” in one hop (See Fig. 5). These approaches reduce trivial subjectpredicate-object (SPO) triples (Lu et al., 2019) significantly, hence, increase the performance of the NLI model for VQA.

heading: 4 EXPERIMENTS

heading: 4.1 DATASETS AND IMPLEMENTATION
text: VQA model: We use two-glimpse BAN (Kim et al., 2018a) for the target VQA model. All hyperparameter settings follow the previous work (Kim et al., 2018a). The input image features are extracted from the Faster-RCNN (Ren et al., 2015), pre-trained using Visual Genome dataset (Krishna et al., 2017). We adaptively extract 10 to 100 objects per image, as in (Kim et al., 2018a). For the question embedding in VQA, we initialize 300 dimension word embeddings with pre-trained GloVe vectors (Pennington et al., 2014).
Explanation Generator: We use the Up-Down captioning model (Anderson et al., 2018b) as our explanation generator. For training the model, we use both train and validation splits of MS COCO captioning dataset (Chen et al., 2015). While building the vocabulary, we drop any words if they occur less than five times in the training dataset, resulting in 10,512 words in ∑ .
VCBS decoding: Since all questions have a different number of constraints |C|, we repeat the VCBS decoding with increasing values of b and p until M is not empty or p = |C|: we initiate with b = 3 and p = 0 (i.e., (b, p) = {(3, 0), (4, 1), (5, 2), (6, 3), (6, 4), · · · , (6, |C|)}). Note that we set an upper limit of b as 6: beam size larger than 6 decreases the decoding speed drastically.
NLI model: We train both the HBMP (Talman et al., 2019) and the ConSeqNet (Wang et al., 2019) on the combination of multiple datasets: SNLI dataset (Bowman et al., 2015), SciTail dataset (Khot et al., 2018), and MultiNLI dataset (Williams et al., 2018). While generating explanations, we only
use the ConSeqNet (Wang et al., 2019) to visualize the linguistic inference process by One-Hop Direct subgraph (See Fig. 6(c)).
Evaluation of VQA Explanations: Park et al. (2018) and Li et al. (2018) evaluated the performance by measuring the similarity between the generated description and its ground truth, using ordinary metrics in image captioning (i.e., CIDEr (Vedantam et al., 2015)). Measuring the CIDEr score for VQA-X or VQA-E is a measure of how well the justified explanation is reproduced. Since we focus on generating explanations through visual and linguistic inference, we use the Mean Opinion Score (MOS) to compare with these methods on one hundred random samples from VQA 2.0 test-std split (He et al., 2016; Antol et al., 2015).
To minimize the bias in the survey, we conduct an online user study in MTurk (Paolacci et al., 2010), using a hundred random samples from VQA 2.0 test-std split. According to our preliminary survey within our research group, survey participants were likely to lost focus on the study if they were asked a hundred questions without a rest. For this reason, we divide 100 questions into five splits (i.e., 20 questions for each split). The study involved 52 Amazon Mechanical Turk workers (Turkers) for each split, who spent an average time of about 12 minutes to complete a split. For each explanation, we ask its score for suitability as reasoning, ranging from 1 to 5 (i.e., 1 indicates a very inappropriate explanation, and 5 corresponds to a very appropriate explanation).

heading: 4.2 QUALITATIVE RESULTS
text: We compare the performance of the proposed method with state-of-the-art methods (PJ-X (Park et al., 2018), Multi-task VQA-E (Multi VQA-E) (Li et al., 2018)), and visualize the difference
between the previous methods and the proposed method in Fig. 6. When the answer to the question is “no,” previous methods only describe the image, but our method clarifies the reason (i.e., the model cannot find shade from the image) (See Fig. 6(b)). Figure 6(c) shows an example when the output of the NLI model is “entailment.” The proposed method first creates a description of the objects and attributes of interest in the image. Then it shows how the answer to the question is inferred from the description.

heading: 4.3 QUANTITATIVE RESULTS
text: We show the MOS results in Table 1. The proposed method attained the highest score, while the PJ-X model (Park et al., 2018) attained similar score as the Multi VQA-E model (Li et al., 2018). We also categorize the MOS scores according to the answer types in Table 2. The result shows that our method for questions of yes/no answers has distinct advantages over the previous methods.
Finally, if the VQA model’s answer is yes/no, we change the answer to “yes” if the output of the NLI model is “entailment” and to “no” otherwise (if p = |C|, we do not change the answer). Table 3 shows the VQA performance comparison of models on the test server submission. The VQA performance is reduced when using the HBMP (Talman et al., 2019) for NLI, but the VQA performance is increased when the ConSeqNet (Wang et al., 2019) is used for NLI. Since the ConSeqNet is trained using broader background knowledge (the HBMP does not use the ConceptNet KG (Speer et al., 2017)), this result shows that background knowledge is essential for answering yes/no in VQA.

heading: 5 CONCLUSION
text: In this paper, we have suggested a novel method that generates explanations for VQA using visual and linguistic inference. Firstly, we describe the relationship between objects and attributes of interest in the image using the proposed VCBS decoding. The VCBS algorithm helps to generate explanations that contain essential concepts from the question. Next, if the answer is yes/no, we use NLI to check whether the content from questions can be inferred from the explanations. If it can be inferred, we visualize a One-Hop Direct subgraph between the explanation and the question. We evaluate the performance of the proposed method by measuring MOS. For the fair comparison, we conduct an online user study using MTurk. The result shows that our model attains better performance than the previous models. Finally, we modified the VQA model’s answer based on the
NLI output and measured the performance. For the state-of-art NLI model (Wang et al., 2019) using ConceptNet KG (Speer et al., 2017), we show an increase in the VQA model’s performance.

heading: A APPENDIX
text: A.1 QUANTITATIVE RESULTS
We used the BUTD model trained on COCO captioning data (Chen et al., 2015) as an explanation generator. The reasons for using it are as follows: 1) The COCO dataset is much larger than the VQA-X (Park et al., 2018) and VQA-E (Li et al., 2018), so we believe that the model trained on the COCO dataset can better understand the relationship between objects in an image. 2) We cannot train the Multi-VQAE (Li et al., 2018) (or PJ-X (Park et al., 2018)) model on the COCO captioning dataset, because the dataset does not include VQA answers. However, we conduct additional experiments to confirm that our method can be used to improve the performance of the existing textual explanation model.
For this purpose, we extract 100 additional experimental samples from the VQA 2.0 test-std split. These samples are chosen so as not to duplicate the samples used in previous experiments. Similar to the experiment in Table 1, we divided 100 samples into five splits. Unlike before, this survey involved 40 AMT workers for each split. Note that we used two different explanation generators in this experiment: the Up-Down (Anderson et al., 2018b) model trained on COCO captioning dataset (Chen et al., 2015), and the Multi-VQAE (Li et al., 2018) trained on VQA-E dataset (Li et al., 2018). The results in Table 4 and Table 5 show that our method helps to generate more appropriate textual explanations, especially when the VQA model’s answer is “yes” or “no.”
A.2 QUALITATIVE RESULTS
We show qualitative results from the experiment conducted in Table 4 (See Fig. 7 and Fig. 8).
A.3 STRICT ENTAILMENT AND WEAK ENTAILMENT
We conducted additional experiments with using the VQA model’s answer as ”yes,” if the class probability of the NLI model for ”Entail” is greater than ”Contradictory,” even though the NLI result is ”Neutral” (Weak Entailment). In this experiment, we found that the answer is often ”yes,” even when there is little correlation between premise and hypothesis. For this reason, we use the answer as ”yes” only if the NLI result is ”Entail” (Strict Entailment) (See Fig. 9).

Review summaries:
The paper proposes a novel method for explaining VQA systems. Different from most previous work, the proposed approach generates textual explanations based on three steps. First, it extracts keywords from the question. Then an explanation sentence is decoded (based on an RNN image captioner) through the proposed Variable-Constrained Beam Search (VCBS) algorithm to satisfy the keyword constraints. Finally, 3) checking through linguistic inference whether the explanation sentence can be used as a premise to infer the question and answer.