Visual question answering aims to answer the natural language question about a given image . Existing graph-based methods only focus on the relations between objects in an image and neglect the importance of the syntactic dependency relations between words in a question . To simultaneously capture the relations between objects in an image and the syntactic dependency relations between words in a question , we propose a novel dual channel graph convolutional network ( DC-GCN ) for better combining visual and textual advantages . The DC-GCN model consists of three parts : an I-GCN module to capture the relations between objects in an image , a Q-GCN module to capture the syntactic dependency relations between words in a question , and an attention alignment module to align image representations and question representations . Experimental results show that our model achieves comparable performance with the state-of-theart approaches . As a form of visual Turing test , visual question answering ( VQA ) has drawn much attention . The goal of VQA ( Antol et al . , 2015 ; Goyal et al . , 2017 ) is to answer a natural language question related to the contents of a given image . Attention mechanisms are served as the backbone of the previous mainstream approaches ( Lu et al . , 2016 ; Yang et al . , 2016 ; Yu et al . , 2017 ) , however , they tend to catch only the most discriminative information , ignoring other rich complementary clues ( Liu et al . , 2019 ) . Recent VQA studies have been exploring higher level semantic representation of images , notably using graph-based structures for better image understanding , such as scene graph generation ( Xu et al . , 2017 ; Yang et al . , 2018 ) , visual relationship detection ( Yao et al . , 2018 ) , object counting ( Zhang et al . , 2018a ) , and relation reasoning ( Cao et al . , 2018 ; Li et al . , 2019 ; Cadene et al . , 2019a ) . Representing images as graphs allows one to explicitly model interactions between two objects in an image , so as to seamlessly transfer information between graph nodes ( e.g. , objects in an image ) . Very recent research methods ( Li et al . , 2019 ; Cadene et al . , 2019a ; Yu et al . , 2019 ) have achieved remarkable performances , but there is still a big gap between them and human . As shown in Figure 1 ( a ) , given an image of a group of persons and the corresponding question , a VQA system needs to not only recognize the objects in an image ( e.g. , batter , umpire and catcher ) , but also grasp the textual information in the question " what color is the umpire 's shirt " . However , even many competitive VQA models struggle to process them accurately , and as a result predict the incorrect answer ( black ) rather than the correct answer ( blue ) , including the state-of-the-art methods . Although the relations between two objects in an image have been considered , the attention-based VQA models lack building blocks to explicitly capture the syntactic dependency relations between words in a question . As shown in Figure 1 ( c ) , these dependency relations can reflect which object is being asked ( e.g. , the word umpire 's modifies the word shirt ) and which aspect of the object is being asked ( e.g. , the word color is the direct object of the word is ) . If a VQA model only knows the word shirt rather than the relation between words umpire 's and shirt in a question , it is difficult to distinguish which object is being asked . In fact , we do need the modified relations to discriminate the correct object from multiple similar objects . Therefore , we consider that it is necessary to explore the relations between words at linguistic level in addition to constructing the relations between objects at visual level . Motivated by this , we propose a dual channel graph convolutional network ( DC-GCN ) to simultaneously capture the relations between objects in an image and the syntactic dependency relations between words in a question . Our proposed DC-GCN model consists of an Image-GCN ( I-GCN ) module , a Question GCN ( Q-GCN ) module , and an attention alignment module . The I-GCN module captures the relations between objects in an image , the Q-GCN module captures the syntactic dependency relations between words in a question , and the attention alignment module is used to align two representations of image and question . The contributions of this work are summarized as follows : 1 ) We propose a dual channel graph convolutional network ( DC-GCN ) to simultaneously capture the visual and textual relations , and design the attention alignment module to align the multimodal representations , thus reducing the semantic gaps between vision and language . 2 ) We explore how to construct the syntactic dependency relations between words at linguistic level via graph convolutional networks as well as the relations between objects at visual level . 3 ) We conduct extensive experiments and ablation studies on VQA-v2 and VQA-CP-v2 datasets to examine the effectiveness of our DC-GCN model . Experimental results show that the DC-GCN model achieves competitive performance with the state-of-the-art approaches . In this paper , we propose a dual channel graph convolutional network to explore the relations between objects in an image and the syntactic dependency relations between words in a question . Furthermore , we explicitly construct the relations between words by dependency tree and align the image and question representations by an attention alignment module to reduce the gaps between vision and language . Extensive experiments on the VQA-v2 and VQA-CP-v2 datasets demonstrate that our model achieves comparable performance with the stateof-the-art approaches . We will explore more complicated object relation modeling in future work .

Challenge: Existing methods for visual question answering only focus on relations between objects in an image neglecting syntactic dependency relations between words in a question.
Approach: They propose a dual channel graph convolutional network which consists of three modules to capture relations within an image, a syntactic dependency, and between modalities.
Outcome: The proposed model achieves comparable performance with the state-of-the-art approaches on VQA-v2 and VQA-CP-v2 datasets.