Before revision:  We observe that "helpful" explanations are conducive to game performance (by almost 22\% for "excellent" rated explanation games), and having at least one "correct " explanation is significantly helpful when VQA system answers are mostly noisy (by almost 30\% compared to no explanation games). We also see that players develop a preference for explanations even when penalized and that the explanations are mostly rated as "helpful" .
After revision:  We observe that "helpful" explanations are conducive to game performance (by almost 22\% for "excellent" rated explanation games), and having at least one "correct " explanation is significantly helpful when VQA system answers are mostly incorrect as compared to no-explanation games. Our experiments, therefore, show that ExAG is an effective means to evaluate the efficacy of AI-generated explanation on human-AI collaborative tasks .