Summary:The paper presents a novel method for answering “How many …?” questions in the VQA datasets.  Unlike previously proposed approaches, the proposed method uses an iterative sequential decision process for counting the relevant entity.  The proposed model makes discrete choices about what to count at each time step.  Another qualitative difference compared to existing approaches is that the proposed method returns bounding boxes for the counted object.  The training and evaluation of the proposed model and baselines is done on a subset of the existing VQA dataset that consists of “How many …?” questions.  The experimental results show that the proposed model outperforms the baselines discussed in the paper. Strengths:1.	The idea of sequential counting is novel and interesting. 2.	The analysis of model performance by grouping the questions as per frequency with which the counting object appeared in the training data is insightful.  Weaknesses:1.	The proposed dataset consists of 17,714 QA pairs in the dev set, whereas only 5,000 QA pairs in the test set.  Such a 3.5:1 split of dev and test seems unconventional.  Also, the size of the test set seems pretty small given the diversity of the questions in the VQA dataset. 2.	The paper lacks quantitative comparison with existing models for counting such as with Chattopadhyay et al.  This would require the authors to report the accuracies of existing models by training and evaluating on the same subset as that used for the proposed model.  Absence of such a comparison makes it difficult to judge how well the proposed model is performing compared to existing models. 3.	The paper lacks analysis on how much of performance improvement is due to visual genome data augmentation and pre-training?  When comparing with existing models (as suggested in above), this analysis should be done, so as to identify the improvements coming from the proposed model alone. 4.	The paper does not report the variation in model performance when changing the weights of the various terms involved in the loss function (equations 15 and 16). 5.	Regarding Chattopadhyay et al. the paper says that “However, their analysis was limited to the specific subset of examples where their approach was applicable.” It would be good it authors could elaborate on this a bit more. 6.	The relation prediction part of the vision module in the proposed model seems quite similar to the Relation Networks,  but the paper does not mention Relation Networks.  It would be good to cite the Relation Networks paper and state clearly if the motivation is drawn from Relation Networks. 7.	It is not clear what are the 6 common relationships that are being considered in equation 1.  Could authors please specify these? 8.	In equation 1, if only 6 relationships are being considered, then why does f^R map to R^7 instead of R^6? 9.	In equations 4 and 5, it is not clarified what each symbol represents, making it difficult to understand. 10.	What is R in equation 15?  Is it reward? Overall:The paper proposes a novel and interesting idea for solving counting questions in the Visual Question Answering tasks.  However, the writing of the paper needs to be improved to make is easier to follow.  The experimental set-up – the size of the test dataset seems too small.  And lastly, the paper needs to add comparisons with existing models on the same datasets as used for the proposed model.  So, the paper seems to be not ready for the publication yet.