Learning to Correction: Explainable Feedback Generation for Visual Commonsense Reasoning Distractor

Authors: jiali  chen
xusen  hei
yuqi  xue
yuancheng  wei
jiayuan  xie
yi  cai
qing  li
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-08 17:06:24

Abstract: Large multimodal models (LMMs) have shown remarkable performance in the visual commonsense reasoning (VCR) task, which aims to answer a multiple-choice question based on visual commonsense within an image. However, the ability of LMMs to correct potential visual commonsense errors in the distractor upon their occurrence is yet under-explored. Drawing inspiration from how a human teacher crafts challenging distractors to test students' comprehension of the concepts or skills and assists them in identifying and correcting errors toward the answer, we are the pioneering research for LMMs to simulate this error correction learning process. To this end, we employ GPT-4 as a ``teacher'' to collect the explainable feedback dataset VCR-DF for error correction, which serves as a benchmark to evaluate the ability of LMMs to identify misconceptions and clarify reasons behind the error in VCR distractors toward final answers. In addition, we propose an LMM-based Pedagogical Expert Instructed Feedback Generation (PEIFG) model to incorporate the learnable expert prompts and multimodal instruction as guidance for feedback generation. Experimental results show that our PEIFG significantly outperforms existing LMMs. We believe our benchmark carves out a new direction for evaluating the capabilities of LMMs.

Main paper: 
 introduction: 
 Visual commonsense reasoning (VCR) task aims to predict the answer to the multiple-choice question and provide a convincing rationale [11, 24, 44, 45, 49] about the image. In recent years, it has gained considerable attention from computer vision (CV) and natural language processing (NLP) communities due to the advancement of large multimodal models (LMMs) [3, 14, 24, 41, 46, 51] . Specifically, inferring a reliable answer in VCR requires LMMs to not only recognize objects and scenes but also deeply understand the underlying visual commonsense (e.g., likely intents, goals, and social dynamics of people) in the image.Scrutinizing existing LMMs [3, 14, 24, 51] , we identify their common paradigm as two stages: pre-training and instruction tuning with a large language model. Specifically, they are first pre-trained on large-scale image-text pairs for modality alignment and then construct instruction data that combines both visual and language features, which is fed into the large language model to infer the answer. By this training paradigm, they often demonstrate strong reasoning and generalization capabilities attributed to the exponential increase in both data size and model scale. In the field of pedagogy [7, 31] , a human teacher often designs challenging distractors to evaluate students' cognitive skills in multiple-choice questions and guide them in identifying and rectifying potential errors toward correct answers. While the capability of LMMs for straightforward reasoning has advanced, we further investigate whether LMMs can emulate the error correction process, which remains unexplored in previous LMM studies. Therefore, we generate feedback as error correction based on the multiple-choice question of VCR, which mainly includes the misconception and explanation about the distractor. However, distractors from the original VCR dataset [49] are inadequate for evaluating the error correction ability of LMMs since the inherent bias in these distractors. Specifically, the sample from the original VCR dataset frequently exhibits a higher overlap between the correct answer and entities than with distractors, where merely choosing the option based on the rule of maximum entity overlap can achieve over 60% accuracy. Moreover, the content of the distractors frequently lacks any pertinent relevance to the image in the original VCR dataset. As shown in Fig. 1 , elements "playdate", "test" and "racing" from VCR dataset have no connection to the image. Consequently, LMMs may easily dismiss these distractors without comprehensively understanding the visual commonsense, leading to a lack of visual commonsense errors for feedback generation.To address the aforementioned issues, we construct the VCR-DF dataset including new distractors and explainable feedback, which can serve as a benchmark to evaluate the error correction ability of LMMs. Specifically, we first utilize the language-only GPT-4 to classify the educational level of the current question with Bloom's taxonomy [2] , which can be categorized into "remember", "understand", "apply", "analyze", "evaluate" and "create" levels. This classification prevents the subsequent generation of distractors from deviating from the corresponding educational level. Next, following [27] , given the input information (i.e., event, question, answer, and objects) and educational level, GPT-4 generates new distractors related to the content of the image, which are subject to further manual screening. Finally, we employ GPT-4 again as a "teacher" to annotate the corresponding misconception and explanation about the distractor. As shown in Fig. 1 , the distractors from our VCR-DF contain visual commonsense errors, which are more relevant to the image. The feedback from VCR-DF includes the educational level, misconception and explanation of the distractor.In this paper, we propose a feedback generation task for VCR distractors to evaluate the error correction ability of LMMs. Diverging from the forward reasoning, LMMs often overlook the potential mismatched image-text information between the distractor and image since they typically learn from image-text pairs during training. Moreover, data specific to error correction is often absent in pre-training datasets, resulting in models' inability to rectify errors when encountered. Experimental results also show that LMMs do not exhibit the same proficiency in error correction as they do in reasoning. Therefore, we argue that equipping LMMs with strong reasoning abilities to perform error correction requires specialized prompts as guidance of the LMM. Building upon this analysis, we introduce Pedagogical Expert Instructed Feedback Generation (PEIFG), an LMM engineered for error correction by feedback generation, which consists of three key components: a visual feature extractor (VFE), an expert prompt selector (EPS), and a text generator. Specifically, the VFE first utilizes the visual marker perceiver (VMP) and CLIP [34] image encoder to extract region-level and global-level visual features, which are concatenated as the image representation. To address the challenge of aligning textual information with multiple instances of the same object category (e.g., person0 and person1) within the image, we add object boxes as visual markers for corresponding objects and then incorporate the SAM-based [20] VMP with a language model (i.e., OPT-350M [50] ) for object coordinates prediction. Subsequently, we develop the EPS to select expert prompts from a learnable prompt pool, representing specialized expert knowledge as guidance for feedback generation. Technically, we combine language instructions with image representations to select the most relevant expert prompts. Finally, we integrate the text prompt, visual features from VFE and expert prompts into the multimodal instruction, which guides the large language model for feedback generation. Furthermore, a refinement step employs reinforcement learning to ensure the generated feedback is explainable and faithful, maintaining logical consistency with the input information. Considering our model effectively identifies visual commonsense errors within distractors, we can further utilize PEIFG to capture these potential errors and generate distractors without modifying the model architecture.Our main contributions can be summarized as follows:• To the best of our knowledge, we are the first to investigate the error correction capabilities of large multimodal models (LMMs).Additionally, we construct a benchmark and introduce the feedback generation task for evaluation.  
related work: 
 Equipping machines with multimodal reasoning ability is a longstanding goal of artificial intelligence (AI) systems [28, 48, 53] . This form of reasoning empowers machines to emulate human-like cognition and commonsense understanding of the world. Recent progress in multimodal learning has been driven by incorporating visual features with pre-trained large language models as large multimodal models (LMMs) [8, 17, 27, 54] . The current paradigm for training LMMs primarily comprises two stages (image-text pre-training and instruction tuning). Specifically, during the imagetext pre-training stage, these LMMs are initially trained on a large scale of image-text pairs for cross-modal alignment [1, 21, 22, 47] . This process ensures that both visual and textual input information are effectively mapped into a unified semantic space. For example, BLIP-2 [22] design a Q-Former to align visual and textual features in the pre-trained stage. During instruction tuning, LMMs fine-tune on multimodal instruction datasets to enhance their instruction-following ability and tackle more complex multimodal tasks. These instruction datasets originate from manually annotated data [5, 9, 15, 30, 38] and data generated by GPT-4 [4, 23, 26, 27, 47] .InstructBLIP [8] builds upon the BLIP-2 framework by fine-tuning Q-Former with instruction data. MiniGPT-4 [54] directly employs a linear layer to project visual features into the semantic space of language, leveraging instruction data for this process. LLaVA [27] is the first LMM to fine-tune through self-instruction, which employs language-only GPT-4 to generate instruction-following data with some manual samples for fine-tuning. Hong et al. [17] propose an 18B LMM CogAgent for graphical user interfaces (GUI) understanding and navigation, which utilizes both low-resolution and high-resolution image encoders. Meanwhile, it achieves the stateof-the-art on various reasoning benchmarks. Moreover, some other works [6, 16, 29, 39] use the large language models as the controller, which aims to control various visual modules with code generation for reasoning. Specifically, VisProg [16] and ViperGPT [39] utilize predefined APIs to access available modules, and compose them by generating Python code for execution without any task-specific training. Existing LMMs primarily focus on forward reasoning capabilities in multimodal tasks. However, their ability to analyze the causes of errors and rectify them is yet under-explored. To the best of our knowledge, we are the first to investigate LMMs for error correction in the visual commonsense reasoning task. 
dataset construction: 
 Existing datasets for visual commonsense reasoning (VCR) [11, 49] are mainly structured as multiple-choice questions. However, they are inadequate for comprehensively assessing models' visual commonsense reasoning capability and their proficiency in error correction. The reasons are as follows: i) The original distractors within these datasets contain inherent bias, where excessive overlap between the entities of the image and answer, and the distractors often lack relevance to the question and image. It results in a deficiency of visual commonsense errors within these distractors. ii) They are deficient in feedback mechanisms to identify and address errors within distractors, which is essential for timely correction.Inspired by cognitive-developmental theory in the field of pedagogy [31] , teachers often employ Bloom's taxonomy [2] to identify questions at various levels and then craft corresponding distracors, aiming to probe students' potential cognitive challenges. Moreover, they provide feedback to assist students in understanding and correcting their errors after failing to solve a problem. Therefore, we construct the VCR-DF dataset as a benchmark for evaluating the error correction ability of large multimodal models (LMMs) in visual commonsense reasoning. In total, the VCR-DF dataset contains 22,401 data samples and splits 20,163 and 2,238 samples for training and testing respectively. Specifically, our VCR-DF dataset is derived from the original VCR dataset [49] , which leverages GPT-4 [32] for the new distractors and feedback data collection. For the input image 𝐼 , question 𝑄, and correct answer 𝐴, we prompt GPT-4 to generate multiple distractors {𝐷 𝑖 } 𝑁 𝑖=1 and corresponding feedback {𝐹 𝑖 } 𝑁 𝑖=1 . The prompts for GPT-4 are shown in the Supplementary. In the following subsection, we will detail the procedure of data collection.To avoid the laborious demand of human annotation, we design a data reformation pipeline assisted by language-only GPT-4 and a manual filtering process for distractor and feedback collection. Specifically, the source images are from the original VCR dataset [49] . We also preserve questions, correct answers and object boxes provided in VCR.For distractor data collection, we guide GPT-4 to identify the Bloom's taxonomy level of the question with the given answer, and generate five distractors with the corresponding Bloom's taxonomy level for each QA pair based on the given inputs (i.e., manually annotated image events and places, questions, answers, and object boxes).Considering the potential mistakes in annotations with GPT-4, we develop a web page for manual distractor filtering to remove those of lower quality. We first ask a group of trained annotators to assess whether each distractor is related to the question and image. Upon confirming relevance, they need to review the distractor against the image, answer, and question for inaccuracies, where distractors without errors are discarded. Finally, annotators rank the distractors and select the top-3 of them.3.1.2 Feedback Collection. Given the above input information and each filtered distractor, we instruct GPT-4 to identify the misconception and explain the error in the distractor, serving as the preliminary feedback data. Subsequently, we manually classify feedback meeting the following quality criteria as the final feedback data. (i) Accuracy: must precisely pinpoint and correct the error within the distractor, ensuring the rectification is directly relevant to the identified mistake. (ii) Clarity: The explanation provided in the feedback should be clear and understandable, avoiding ambiguity or overly complex language that could hinder comprehension. After our manual evaluation, over 90% of the preliminary samples are both meet the above two criteria. Finally, we integrate synthetic Bloom's taxonomy levels, misconceptions and explanations about the error as feedback data, as shown in Fig. 1 . 
methodology: 
 Our goal is to develop a large multimodal model (LMM) for explainable feedback generation, which simulates the teaching process of educators. Specifically, given the input image 𝐼 , question 𝑄, correct answer 𝐴, and 𝑁 distractors {𝐷 𝑖 } 𝑁 𝑖=1 , we identify the corresponding Bloom's taxonomy level of the question, and then generate the misconception and explanation as feedback {𝐹 𝑖 } 𝑁 𝑖=1 . The overall architecture of our proposed Pedagogical Expert Instructed Feedback Generation (PEIFG) model is shown in Fig. 2 , which consists of three components: (i) visual feature extractor (VFE), which adds the object box information as visual markers into the image and obtains the contextually enriched visual features. (ii) expert prompt selector (EPS), which incorporates language instructions and visual features to select the most relevant expert prompts as expert knowledge from a learnable prompt pool. (iii) text generator, which constructs a multimodal instruction for the large language model (LLM), integrating the visual features, expert prompts and language instruction to generate the feedback. The details of each component are shown in the following subsections.Different from previous visual question answering datasets (e.g., VQA v2.0, OKVQA and so on), where the input is an image and question. The VCR dataset provides the model with additional object boxes including multiple instances of the same object category (e.g., person0 and person1). Consequently, we first automatically annotate the objects within the image with the provided boxes as visual markers, as shown in Fig. 2 . Then, we introduce a visual marker perceiver (VMP) to comprehend the visual markers information (i.e., object boxes and object text annotation) in the image by stage 1 training, as shown in Fig. 2 . The region-level features from the VMP and global-level features from CLIP [34] image encoder are concatenated as final visual features.Given the image with visual markers, we enable the visual marker perceiver (VMP) to understand the object with markers in stage 1 training. Specifically, we initially reshape the image into high resolution (i.e., 1024 × 1024) and use the SAM-base [20] backbone with two convolution layers as VMP to obtain the region-level visual features.Following [43] , we fed the region-level features and detection instruction into a language model (i.e., OPT-350M) [50] to effectively enhance the VMP capability in discerning object spatial information through visual markers. Technically, we employ a multilayer perception (MLP) to map region-level visual features into the semantic space of language. For the input of OPT model, we fill them into the pre-defined detection instruction template, i.e., "<img> Detect all objects in the image". The special token "<img>" is replaced by the region-level visual features. The coordinates of the object boxes are provided for prediction by the OPT model. Specifically, we normalize the coordinates of top-left and bottom-right corners to the range of [0, 1] according to the image size, which provides the OPT [50] model for prediction. The language modeling loss is utilized to optimize both the VMP and OPT models concurrently, facilitating object perception with visual markers.4.1.2 Visual Features Extraction. After stage 1 training, we employ the trained VMP and CLIP image encoder to obtain regionlevel and global-level features, respectively. Specifically, given the high-resolution image with visual markers 𝐼 ℎ , VMP extracts the region-level visual features. Meanwhile, we utilize the CLIP image encoder [34] to encode the low-resolution image 𝐼 𝑟 ∈ R 224×224 , and then obtain the global-level visual features. Subsequently, we map both types of visual features into the semantic space of language, enhancing their compatibility with the input format of the large language model for further feedback generation, denoted as 𝑣 𝑟 and 𝑣 𝑔 respectively. The process of visual feature extraction can be formulated as:EQUATIONwhere 𝑓The introduction of the expert prompt selector (EPS) is motivated by the desire to enable our model to emulate the diverse expertise of teachers as expert knowledge in generating feedback. Thus, we maintain a learnable prompt pool representing diverse expert knowledge and employ instruction-aware visual features (i.e., integrating both visual and language instruction) to select the relevant expert prompts from the pool.Instruction-aware Visual Feature. We design manually crafted natural language instructions to align with the integrated visual features for further expert prompt selection. Additionally, we supplement these instructions with the corresponding question and answer as auxiliary information. Following [8] , we use the Query Transformer (Q-Former) [22] module to extract the instructionaware visual features. Technically, within the Q-Former, learnable query tokens align with the language instructions by self-attention mechanisms and align with the integrated visual features by crossattention mechanisms. Next, the output sequence of Q-Former is fed into an average-pooling layer to obtain instruction-aware visual features 𝑣 𝑠 . The computation of this process can be expressed as:EQUATIONwhere 𝑣 𝑠 ∈ R 768 . 𝑋 𝑞 ∈ R 𝐿 𝑞 ×768 is the learnable query tokens and 𝑋 𝑛 ∈ R 𝐿 𝑛 ×768 denotes embeddings of the manually crafted instruction. 𝐿 𝑞 and 𝐿 𝑛 correspond to the lengths of the query tokens and instruction, respectively. Q-Former(•) and Avg(•) represent the Q-Fromer module and average-pooling layer respectively. To select the expert prompts relevant to the current input information (i.e., the image, question and answer), we use the instruction-aware visual features as guidance for selection. We first build a learnable prompt pool to maintain diverse expert knowledge which can be formulated as:EQUATIONwhere 𝑃 𝑖 ∈ R 𝐿 𝑝 ×768 and 𝐿 𝑝 is the length of each expert prompt. 𝑆 denotes the size of the prompt pool. Given the necessity for the pool to encompass a wide array of expert prompts, we consider that each prompt should retains its distinct expertise. Each prompt should be jointly independent in the sense that each of them contains unique information. To this end, we design the expert correlation loss to reduce the correlation among the expert prompts in the pool by minimizing their inner product:EQUATIONwhere diag(•) only preserves diagonal entries and || • || 2 𝐹 is the square of the Frobenius norm. Subsequently, we employ the query and match key-value based strategy for expert prompt selection. Concretely, we utilize the average pooling result of each expert prompt as its corresponding key, denoted as {𝑘 1 , 𝑘 2 , ..., 𝑘 𝑆 }, where 𝑘 𝑖 ∈ R 768 . Importantly, the keys are updated corresponding to the expert prompts during training. Given the input information, we aim to find out the top-𝐾 keys {𝑘 𝑚 1 , 𝑘 𝑚 2 , ..., 𝑘 𝑚 𝐾 }, making them closer to the input sample, where {𝑚 𝑗 } 𝐾 𝑗=1 denotes a subset of 𝐾 (1 ≤ 𝐾 ≤ 𝑆) indices of keys in the pool. Specifically, we calculate the cosine similarity 𝑠𝑖𝑚(•) between keys and the instruction-aware visual features 𝑣 𝑠 to select top-𝐾 keys and calculate the key matching loss for selection:EQUATIONEQUATIONwhere 𝑘 is the set of top-𝐾 keys. Finally, we obtain the top-𝐾 expert prompts from the pool corresponding to the selected keys:EQUATIONwhere P ∈ R (𝐾 ×𝐿 𝑝 ) ×768 , 𝑚 𝑖 denotes indices of selected prompts P and [; ] is the concatenation operation.Upon acquiring the integrated visual features and top-𝐾 expert prompts, we fuse them into the LLM-based text generator (i.e., QWen1.5) for feedback generation. Specifically, we adopt the widely used instruction tuning method to incorporate the language prompt, integrated visual features, and expert prompts into multimodal instruction. We first utilize a multilayer perception (MLP) to project the expert prompts into a 2𝑑 dimensional space, meeting the input requirement of the LLM. The multimodal instruction is defined as: "Image: <img>. Expert: <expert>. Please generate the feedback based on the question: {Question}, answer: {Answer}, distractor: {Distractor}". The special tokens "<img>" and "<expert>" are replaced by integrated visual features and selected expert prompts, respectively. "{Question}", "{Answer}" and "{Distractor}" are the input question, answer and distractor of a specific sample. The multimodal instruction is directly fed into the frozen large language model with learnable LoRA layers [18] for language modeling. The cross-entropy loss of language modeling can be formulated as:EQUATIONwhere log 𝑝 𝜃 (•) is the negative log-likehood, Ins is the multimodal instruction, and 𝑤 <𝑡 represents the words before the 𝑡-th word.Given our method's proficiency in identifying visual commonsense errors in distractors, we can utilize the PEIFG model to further capture these potential errors and generate distractors.Refinement. To ensure the logical coherence between the feedback and input information, we leverage the trained PEIFG model to generate pseudo training feedback data, which is then used to refine the model performance with the assistance of GPT-4. Specifically, we randomly sample from the VCR-DF training data and employ the top-p sampling method to generate 𝑀 feedback instances for each sample, denoted as { F1 , F2 , ..., F𝑀 }. Given the question, ground truth distractor, feedback, and the generated feedback, we formulate five diagnostic questions for GPT-4 to ascertain whether the generated feedback meets the specified criteria. For each diagnostic question, feedback that meets the criteria is awarded 1 point, otherwise 0 point. Therefore, the final diagnostic score 𝑠 𝑑 for each generated feedback ranges from 0 to 5. Finally, we rank the generated feedback as pairs based on the diagnostic score and design a refinement loss L 𝑟𝑒 , which utilizes the direct preference optimization (DPO) [36] as reinforcement learning algorithm to further optimize the LLM. More details about the refinement process are shown in the Supplementary.In our setting, we treat the generation of feedback as two distinct tasks. Specifically, feedback generation, we generate feedback for error correction for each given distractor within the VCR-DF dataset. The objective of our PEIFG model is to minimize the total loss, i.e., key matching loss in Eq. 6, correlation loss in Eq. 4 and language modeling loss in Eq. 8. The definition of total loss is:EQUATIONwhere 𝐷 is the total number of training samples, 𝜆 1 and 𝜆 2 stand for hyperparameters. It is worth noting that the refinement loss L 𝑟𝑒 is applied to optimize the model's parameters after the PEIFG has been trained. 
experiment 5.1 implementation details: 
 We implement our PEIFG model with Pytorch and train it on two RTX 3090 cards. For the visual feature extraction, we employ the SAM-base [20] backbone and two convolution layers as the visual marker perceiver (VMP). Furthermore, in this stage 1 training for the VMP module, we use AdanW [19] optimizer with an initial learning rate of 5e-5. The ViT-L/14 [12] pre-trained in CLIP [34] is used for the image encoder, where ViT-L/14 denotes ViT-Large model with the patch size 14 × 14. Thus, the length of image tokens 𝐿 𝑑 = 256 and the dimension of features 𝑑 is 1024. When selecting the expert prompt, we set the size of the prompt pool 𝑆 to 10 and the length of each expert prompt 𝐿 𝑑 to 5. The number of selected prompts 𝐾 is 3. The application of expert prompts differs in distractor and feedback generation. Specifically, we leverage top-3 expert prompts, which allows for a more comprehensive analysis by incorporating diverse expert knowledge into multimodal instruction. For distractor generation, within single forward, we aim to generate 3 distractors. To achieve this, each of the three expert prompts is utilized to generate one corresponding distractor, ensuring a tailored approach to leverage the distinct expertise of each expert prompt. We choose the QWen1.5-1.8B as the LLM for distractor and feedback generation. The LoRA layers are inserted into each self-attention layer of the LLM for optimization. During training, we use Adam optimizer with cosine scheduler and the initial learning rate of 8e-5 to optimize the total loss function L.We only fine-tune the learnable query tokens of Q-Former, expert prompt pool, multilayer perception (MLP) and LoRA layers of LLM, while freezing other parameters. For LoRA, we set the rank to 8. The batch size is 24 and we train 3 epochs. The hyperparameters 𝜆 1 and 𝜆 2 for the total loss function in Eq. 9 are both 0.1.5.2.1 Baseline Models. In this paper, we evaluate our proposed PEIFG by comparing it with two types of baseline models.• Explanation-Enhanced visual question answering models including NLX-GPT [37] and KICNLE [42] . Specifically, NLX-GPT adopts the pre-trained CLIP encoder and GPT-2 [35] language model for feedback and distractor generation. KICNLE incorporates external knowledge and designs a multi-iteration generative approach to ensure logical consistency between the generated distractor and feedback. • Multimodal large language models (LMMs) with different parameter scales from 3B to 18B, including BLIP-2 [22] , Instruct-BLIP [8] , VisualGLM [13] , LLaVA-v1.5 [27] and CogAgent [17] . Specifically, we integrate the LoRA layers into the self-attention mechanisms of LLMs. Furthermore, we randomly select 200 samples for comparison between our model and the GPT-4 with vision (GPT-4V) [32] , which boasts parameter scales above 175B and has the best performance on various multimodal tasks.More implementation details of baseline models are provided in the Supplementary.To investigate the performance effect of each module in PEIFG, we compare the following variants of our method on VCR-DF dataset. We independently conduct corresponding ablation experiments for each module. [25] , METEOR [10] , CIDEr [40] , and BERTScore [52] . These metrics are commonly used for evaluating text generation and we compute metric values using the publicly available code 2 . iv) Finally, we investigate the impact of the refinement operation assisted bv GPT-4 on feedback generation. After refinement, we find this step can improve the performance on all metrics, which further ensures the logical coherence between the feedback and input information. In particular, the prediction accuracy of educational levels is improved from 86% to 88%.Fig. 3 shows the feedback generated by VisualGLM, CogAgent and our model PEIFG. Intuitively, PEIFG generates more grounded feedback compared to other baseline models. Specifically, we find that:i) The feedback generated by VisualGLM frequently lacks crucial information in its explanations that would lead to the correct answer. Specifically, as shown in Fig. 3 , the explanation generated by VisualGLM incorrectly classifies the educational level and recognizes the location of person9 as being 'in the ring' rather than the actual "by the ring". Worse still, it merely states the reason for the error without further elaborating on the correct answer that "person9 is a commentator". In contrast, both CogAgent and our PEIFG accurately recognize the location of "person9" and correctly conclude that "person9 is a commentator". ii) Compared our PEIFG with CogAgent, the explanations generated by CogAgent overlook the visual clue 'mic', which may hinder to logically deduce the correct answer that 'person9 is a commentator'. Although CogAgent contains eight times the number of parameters as our PEIFG model, our model still manages to more accurately capture visual clues "by the ring" and "mic" for error correction. This can be attributed to the efficacy of our expert prompts, which effectively instruct the LLM to focus more on visual clues relevant to error correction. 
conclusion: 
 In this paper, we present a pioneering work to investigate the error correction capabilities in visual commonsense reasoning of existing large multimodal models (LMMs), noted for their exceptional reasoning skills. Therefore, we leverage GPT-4 to construct the VCR-DF benchmark for evaluation of error correction by feedback generation task. Additionally, we propose the PEIFG model to identify visual commonsense errors and provide explainable feedback. Specifically, we first design a visual feature extractor to obtain the integrated visual features. Subsequently, the integrated visual features and language instruction are jointly used to select the relevant expert prompts from the pool. Finally, we effectively integrate the visual features and learnable expert prompts into multimodal instruction and design a refinement strategy for feedback generation.The experimental results indicate that our PEIFG outperforms existing LMMs. We hope this learning to correction process would chart a new direction to evaluate the capabilities of LMMs. 
 
 Appendix:  


Decision: Overall, this paper is innovative with abundant experimental content. The authors have effectively addressed the reviewers' questions, and we agree to accept this paper.

Reviews: review: paper_summary: paper_summary: This article introduces a novel approach to enhance Large Multimodal Models' (LMMs) capabilities in correcting visual commonsense reasoning (VCR) errors within distractors, inspired by the method a human teacher uses to craft distractors for testing and educating students. The article pioneers the use of GPT-4 as a "teacher" to compile an explainable feedback dataset named VCR-DF, aimed at benchmarking LMMs' abilities to identify and explain errors in VCR distractors, leading to the correct answers. Furthermore, the research proposes the Pedagogical Expert Instructed Feedback Generation (PEIFG) model, which integrates learnable expert prompts and multimodal instructions to guide feedback generation. The experimental findings indicate that PEIFG significantly surpasses existing LMMs in performance, suggesting a new direction for evaluating LMMs' capabilities.
strength_weakness: strength_weakness: This article presents several strengths in its approach to enhancing the error correction capabilities of large multimodal models (LMMs) through the proposed Pedagogical Expert Instructed Feedback Generation (PEIFG) framework. 
1.Innovative Integration of Visual and Textual Feedback: The PEIFG model introduces a novel method for combining visual features extracted using a visual feature extractor (VFE), which includes a Visual Marker Perceiver (VMP) and CLIP image encoder. 
2.Customized Expert Prompt Selection for Enhanced Error Correction: The development of an Expert Prompt Selector (EPS) within the PEIFG model represents a significant advancement. By selecting the most relevant expert prompts from a learnable prompt pool, the model can provide specialized guidance for feedback generation. 
3. Refinement and Evaluation through Reinforcement Learning: The inclusion of a refinement step that employs reinforcement learning to improve the explainability and faithfulness of the generated feedback is a crucial advantage.
limitations: limitations: 1.	The different representations of VMP in Figure 2 may cause confusion. (Training or Freezing)
2.	Why train both VPM and OPT models simultaneously? Is it necessary to train the OPT model?
3.	The Expert Prompt Selection needs more theoretical explanation. At the same time, more detailed ablation experiments are needed to analyze the role of the internal components of this module, such as the ablation experiment of top_k.
4.	The section title "5.4.1 performance Comparison" should be changed to "5.4.1 Performance Comparison".
paper_summary: paper_summary: This paper aimed to explore the ability of Large Multimodal Models (LMMs) in correcting potential visual commonsense errors in distractor and proposed a dataset VCR-DF to tackle this.
strength_weakness: strength_weakness: 1, This paper constructed the VCR-DF dataset to evaluate the ability of LMMs in clarifying the reasons behind errors in VCR distractors. 
2, To engage the LMM with strong reasoning abilities in error correction, they propose the Pedagogical Expert Instructed Feedback Generation (PEIFG) mode.
3, Experimental results demonstrate that the paper proposed Pedagogical Expert Instructed Feedback Generation (PEIFG) model significantly outperforms existing LMMs.
limitations: limitations: 1, What is the difference of the task and asking the model to describe why the correct answer was chosen? 
2, In the methods section, the correct answer is provided to the model, and then generates feedback for distractor. Could the introduction of correct answer offer additional information? If the correct answer is absent, would the model still maintain good performance?
3, Figure 2 is unclear, especially regarding the VFE. I don’t understand why only Stage 1 training is shown. Where is Stage 2?
4, All the grounding feedbacks in VCR-DF are generated by GPT-4. Why are the results of the proposed model still higher than those of GPT-4 in Table 2?
paper_summary: paper_summary: This paper introduces a feedback generation task for the visual commonsense reasoning distractors to evaluate the error correction ability of large multimodal models (LMMs). In particular, the paper constructs the VCR-DF dataset including new distractors and explainable feedback, which can serve as a benchmark to evaluate the error correction ability of LMMs. Besides, the paper devises a pedagogical expert instructed feedback generation model to conduct the feedback generation. Extensive experiments are conducted to verify the effectiveness of the proposed model.
strength_weakness: strength_weakness: Strong points:
1. The paper aims to investigate the error correction ability of LMMs, which is vital and promising.
2. The paper provides a comprehensive survey of related studies.
3. The paper provides figures to show the proposed model, which is vivid and can facilitate understanding.
limitations: limitations: Weak points:
1. The paper utilizes both the region-level and global-level features, but does not conduct ablation studies to verify the necessity.
2. The paper is suggested to provide a more detailed description of the VCR-DF dataset.
3. I wonder if the constructed VCR-DF dataset can be released to facilitate other researchers.
paper_summary: paper_summary: This paper presents to address the ```distractor`` problem in visual commonsense reasoning.
In this vein, this work makes the following two contributions:
First, the authors collect a new dataset based on the original VCR dataset.
The dataset curation leverages the generation of GPT-4.
Second, the authors propose a method to address this problem.
The experiments on this new dataset prove the effectiveness of the proposed method.
strength_weakness: strength_weakness: - The authors propose to leverage GPT-4 to generate distracted sentences, which is interesting and novel.
- This paper contains extensive experiments on the new dataset especially shown in Table 1.
We can see the authors re-implement so many baselines and produce their results on this new dataset.
limitations: limitations: - I'm confused about the key definition of this task.
It seems we do not have to limit this problem to visual commonsense reasoning.
In particular, the dataset itself has some flaws and does not involve much ```visual commonsense```.
As a result, the generated distractors do not demonstrate this virtue as well.
- The proposed method is too complicated to understand.
It leads to two serious problems: 
1) it is hard for readers to distinguish which parts are novel and are designed by this work.
2) It is almost impossible to reproduce the results of this work.
- The authors leverage a framework from the Pedagogical domain.
To me, this will 'distract' the key contribution of this work as it generally introduces many difficulties to grasp the key idea and there is scanty connection between this work and Pedagogy.