ea VQA : An Experimental Analysis on Visual Question Answering Models (2021)

Abstract: Visual Question Answering (VQA) has recently become a popular research area. VQA problem lies in the boundary of Computer Vision and Natural Language Processing research domains. In VQA research, the dataset is a very important aspect because of its variety in image types i.e. natural and synthetic and also question answer source i.e. originated from human source or computer-generated question answer. Various details about each dataset is given in this paper, which can help future researchers to a great extent. In this paper, we discussed and compared the experimental performance of Stacked Attention Network Model (SANM) and bidirectional LSTM and MUTAN based fusion models. As per the experimental results, MUTAN accuracy and loss are 29% and 3.5 respectively. SANM model is giving 55% accuracy and a loss of 2.2 whereas VQA model is giving 59% accuracy and 1.9 loss.

## introduction the visual question answer model should have the ability to understand both visual and linguistic capabilities. these models must possess some capabilities to function. they are locating an object (finding the location of any object within any given image mostly based on coordinate position along with height and width), finding object attributes (retrieval of attributes of any given object, e.g., color, shape, or any other traits of an object), activity being performed by an object (e.g., sport being played, running, walking, etc.), understanding of any given scene which is basically providing a high-level representation of the environment. the vqa models are prone to bias to remove this problem; multiple answers are normally being provided for any question. this paper has been structured in the following way. section 2 will give related existing work or literature study. this will be followed by section 3, which will give a study of existing datasets available which is relevant for this analysis along with dataset details like licensing, year of formation, and other statistics about image, question, answer, etc. section 4 will give details about the experiment being conducted along with an analysis of the results. section 5 has conclusion details. ## background study assessing the performance of the vqa model is a bit tricky. regular ml models metrics like accuracy (for classification problems), mae (for regression problems), etc., cannot estimate performance properly. to effectively quantify the performance of a vqa model, the metric @xcite must have the following capabilities: a. consistency: this depicts the ability to provide a consistent answer for a different form of the same question. b. grounding/localization: the main purpose is better model interpretability. this is the ability to localize the region in the image which is relevant with respect to the answer. c. plausibility: providing a justifiable answer, e.g., for a question like, is it a sunny day? the answer will be either yes, no, or cannot say something of this like. vqa models generate an attention map to perform object localization with respect to the question being asked. the attention maps are nothing but a matrix that identifies a region within an image that is relevant for an image-question pair.a ranking score is generated based on a number of overlaps of bounding regions @xcite . in a simple attention map mechanism mix of images and questions can derive whether a region is relevant or not with respect to that image. they propose a new architecture that represents the ## related datasets this section is focused on the different dataset descriptions. dataset is an important part of vqa. ## experimental result and performance analysis for the experiment, we have used 30 epochs with minibatch gradient descent with a batch size of 128. each epoch took almost 1.5 hours for the stacked area network model and roughly 2 hours for the vqa model.now we will discuss internal model details.