Original: Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering
Revised: Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering

Original: Anonymous submission ID:
Revised: Anonymous submission ID:

Original: Abstract
Revised: Abstract

Original: When answering a question, humans utilize the information available across different modalities to synthesize a consistent and complete chain of thought (CoT).
Revised: When answering a question, humans utilize the information available across different modalities to synthesize a consistent and complete chain of thought (CoT).

Original: Thisprocess is normally a black box in the case of deep learning models like large-scalelanguage models.
Revised: Thisprocess is normally a black box in the case of deep learning models like large-scalelanguage models.

Original: Recently, science question benchmarks have been used to diagnose the multi-hop reasoning ability and interpretability of an AI system.
Revised: Recently, science question benchmarks have been used to diagnose the multi-hop reasoning ability and interpretability of an AI system.

Original: However,existing datasets fail to provide annotations for the answers, or are restricted tothe textual-only modality, small scales, and limited domain diversity.
Revised: However,existing datasets fail to provide annotations for the answers, or are restricted tothe textual-only modality, small scales, and limited domain diversity.

Original: To this end,we present Science Question Answering ( SQA ), a new benchmark that consists of⇠ 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations.
Revised: To this end,we present Science Question Answering ( SQA ), a new benchmark that consists of⇠ 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations.

Original: We further design language models to learn to generate lectures and explanations as thechain of thought (CoT) to mimic the multi-hop reasoning process when answering
Revised: We further design language models to learn to generate lectures and explanations as thechain of thought (CoT) to mimic the multi-hop reasoning process when answering

Original: SQA questions.
Revised: SQA questions.

Original: SQA demonstrates the utility of CoT in language models, as CoTimproves the question answering performance by 1.20% in few-shot GPT-3 and3.99% in ﬁne-tuned UniﬁedQA.
Revised: SQA demonstrates the utility of CoT in language models, as CoTimproves the question answering performance by 1.20% in few-shot GPT-3 and3.99% in ﬁne-tuned UniﬁedQA.

Original: We also explore the upper bound for models toleverage explanations by feeding those in the input; we observe that it improvesthe few-shot performance of GPT-3 by 18.96%.
Revised: We also explore the upper bound for models toleverage explanations by feeding those in the input; we observe that it improvesthe few-shot performance of GPT-3 by 18.96%.

Original: Our analysis further shows thatlanguage models, similar to humans, beneﬁt from explanations to learn from fewerdata and achieve the same performance with just 40% of the data.
Revised: Our analysis further shows thatlanguage models, similar to humans, beneﬁt from explanations to learn from fewerdata and achieve the same performance with just 40% of the data.

Original: (cid:52)(cid:88)(cid:72)(cid:86)(cid:87)(cid:76)(cid:82)(cid:81) (cid:29)(cid:3)(cid:58)(cid:75)(cid:76)(cid:70)(cid:75)(cid:3)(cid:87)(cid:92)(cid:83)(cid:72)(cid:3)(cid:82)(cid:73)(cid:3)(cid:73)(cid:82)(cid:85)(cid:70)(cid:72)(cid:3)(cid:73)(cid:85)(cid:82)(cid:80)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)
Revised: (cid:52)(cid:88)(cid:72)(cid:86)(cid:87)(cid:76)(cid:82)(cid:81) (cid:29)(cid:3)(cid:58)(cid:75)(cid:76)(cid:70)(cid:75)(cid:3)(cid:87)(cid:92)(cid:83)(cid:72)(cid:3)(cid:82)(cid:73)(cid:3)(cid:73)(cid:82)(cid:85)(cid:70)(cid:72)(cid:3)(cid:73)(cid:85)(cid:82)(cid:80)(cid:3)(cid:87)(cid:75)(cid:72)(cid:3)

Original: [Table]
Revised: [Table]

Original: dataset where a data example consists of multimodal question answering information and the grounded lecture and explanation.
Revised: dataset where a data example consists of multimodal question answering information and the grounded lecture and explanation.

Original: We study if models can generate a reasonable explanation to reveal the chain-of-thought reasoning when answering an SQA question
Revised: We study if models can generate a reasonable explanation to reveal the chain-of-thought reasoning when answering an SQA question

Original:  Introduction
Revised:  Introduction

Original: A long-standing goal of AI systems is to act reliably and learn complex tasks efﬁciently like humanbeings.
Revised: A long-standing goal of AI systems is to act reliably and learn complex tasks efﬁciently like humanbeings.

Original: In the process of reliable decision making, humans follow an explicit chain-of-thought (CoT)reasoning process that is typically expressed as an explanation.
Revised: In the process of reliable decision making, humans follow an explicit chain-of-thought (CoT)reasoning process that is typically expressed as an explanation.

Original: However, machine learning modelsare trained mostly using a large number of input-output examples to perform a speciﬁc task.
Revised: However, machine learning modelsare trained mostly using a large number of input-output examples to perform a speciﬁc task.

Original: Theseblack-box models only generate the ﬁnal decision without reliably revealing the underlying reasoning process.
Revised: Theseblack-box models only generate the ﬁnal decision without reliably revealing the underlying reasoning process.

Original: Not surprisingly, it is unclear if they understand the task and can generalize even thoughthey perform well on the benchmark.
Revised: Not surprisingly, it is unclear if they understand the task and can generalize even thoughthey perform well on the benchmark.

Original: On the other hand, humans are able to learn from instructionsor explanations from past experience and generalize them to novel and unseen problems.
Revised: On the other hand, humans are able to learn from instructionsor explanations from past experience and generalize them to novel and unseen problems.

Original: This helpsthem learn more quickly with fewer data.
Revised: This helpsthem learn more quickly with fewer data.

Original: In this work, we explore if machines can be endowed withsuch reasoning abilities in the context of science-based question answering.
Revised: In this work, we explore if machines can be endowed withsuch reasoning abilities in the context of science-based question answering.

Original: Recently, science problem solving benchmarks [17] have been used to diagnose the multi-hopreasoning ability and interpretability of AI systems.
Revised: Recently, science problem solving benchmarks [17] have been used to diagnose the multi-hopreasoning ability and interpretability of AI systems.

Original: To answer science questions, a model needs tonot only understand multimodal contents but also extract external knowledge to arrive at the correctanswer.
Revised: To answer science questions, a model needs tonot only understand multimodal contents but also extract external knowledge to arrive at the correctanswer.

Original: Since these tasks require domain-speciﬁc knowledge and explicit multi-hop reasoning, amodel would be not interpretable if it fails to provide explanations to reveal the reasoning process.
Revised: Since these tasks require domain-speciﬁc knowledge and explicit multi-hop reasoning, amodel would be not interpretable if it fails to provide explanations to reveal the reasoning process.

Original: However, current science question datasets [17, 16, 46] mostly lack annotated explanations for theanswers.
Revised: However, current science question datasets [17, 16, 46] mostly lack annotated explanations for theanswers.

Original: To address this issue, other science datasets annotate the explanations, but they are restrictedto the textual only modality and limited to small data scales [12, 7, 33] or a small set of topics [19, 13].
Revised: To address this issue, other science datasets annotate the explanations, but they are restrictedto the textual only modality and limited to small data scales [12, 7, 33] or a small set of topics [19, 13].

Original: Therefore, we collect Science Question Answering ( SQA ), a large-scale multi-choice dataset thatcontains multimodal science questions with explanations and features rich domain diversity.
Revised: Therefore, we collect Science Question Answering ( SQA ), a large-scale multi-choice dataset thatcontains multimodal science questions with explanations and features rich domain diversity.

Original: SQA is collected from elementary and high school science curricula, and contains 21,208 examplesalong with lectures and explanations.
Revised: SQA is collected from elementary and high school science curricula, and contains 21,208 examplesalong with lectures and explanations.

Original: Different from existing datasets [16, 17, 46], SQA has richerdomain diversity from three different subjects: natural science, social science, and language science.
Revised: Different from existing datasets [16, 17, 46], SQA has richerdomain diversity from three different subjects: natural science, social science, and language science.

Original: A typical SQA example consists of a question, multiple choices, visual and textual contexts, a correctanswer, as well as a lecture and an explanation.
Revised: A typical SQA example consists of a question, multiple choices, visual and textual contexts, a correctanswer, as well as a lecture and an explanation.

Original: The lecture and explanation provide general externalknowledge and speciﬁc reasons, respectively, for arriving at the correct answer.
Revised: The lecture and explanation provide general externalknowledge and speciﬁc reasons, respectively, for arriving at the correct answer.

Original: Consider the thoughts one person might have when answering the question in Figure 1.
Revised: Consider the thoughts one person might have when answering the question in Figure 1.

Original: One ﬁrstrecalls the knowledge regarding the deﬁnition of a force learned from textbooks: “ A force is a push ora pull that ...
Revised: One ﬁrstrecalls the knowledge regarding the deﬁnition of a force learned from textbooks: “ A force is a push ora pull that ...

Original: The direction of a push is ...
Revised: The direction of a push is ...

Original: The direction of a pull is ... ”, then forms a line of reasoning:“ The baby’s hand applies a force to the cabinet door . !
Revised: The direction of a pull is ... ”, then forms a line of reasoning:“ The baby’s hand applies a force to the cabinet door . !

Original: This force causes the door to open . !
Revised: This force causes the door to open . !

Original: Thedirection of this force is toward the baby’s hand . ”, and ﬁnally arrives at the correct answer: “ Thisforce is a pull . ”.
Revised: Thedirection of this force is toward the baby’s hand . ”, and ﬁnally arrives at the correct answer: “ Thisforce is a pull . ”.

Original: Following [36], we formulate the SQA task to output a natural explanation alongsidethe predicted answer.
Revised: Following [36], we formulate the SQA task to output a natural explanation alongsidethe predicted answer.

Original: In this paper, we train language models to generate lectures and explanations asthe chain of thought (CoT) to mimic the multi-hop reasoning process to answer SQA questions.
Revised: In this paper, we train language models to generate lectures and explanations asthe chain of thought (CoT) to mimic the multi-hop reasoning process to answer SQA questions.

Original: Our experiments show that current multimodal methods [49, 1, 20, 9, 24, 31] fail to achieve satisfactory performance on SQA and do not generate correct explanations.
Revised: Our experiments show that current multimodal methods [49, 1, 20, 9, 24, 31] fail to achieve satisfactory performance on SQA and do not generate correct explanations.

Original: However, we ﬁnd that CoT canhelp large language models not only in the few-shot learning setting but also in the ﬁne-tuning setting.
Revised: However, we ﬁnd that CoT canhelp large language models not only in the few-shot learning setting but also in the ﬁne-tuning setting.

Original: When combined with CoT to generate the lecture and explanation, the ﬁne-tuned UniﬁedQA [18]achieves an improvement of 3.99% as opposed to not using CoT in the ﬁne-tuning stage.
Revised: When combined with CoT to generate the lecture and explanation, the ﬁne-tuned UniﬁedQA [18]achieves an improvement of 3.99% as opposed to not using CoT in the ﬁne-tuning stage.

Original: The few-shot
Revised: The few-shot

Original: GPT-3 model [5] via chain-of-thought prompting can obtain 75.17% on SQA with an improvementof 1.20% compared to the few-shot GPT-3 without CoT. Prompted with CoT, GPT-3 can generatereasonable explanations as evaluated by automated metrics, and promisingly, 65.2% of explanationsmeet the gold standard of human evaluations.
Revised: GPT-3 model [5] via chain-of-thought prompting can obtain 75.17% on SQA with an improvementof 1.20% compared to the few-shot GPT-3 without CoT. Prompted with CoT, GPT-3 can generatereasonable explanations as evaluated by automated metrics, and promisingly, 65.2% of explanationsmeet the gold standard of human evaluations.

Original: We also investigate the upper bound for models toharness explanations by including them in the input.
Revised: We also investigate the upper bound for models toharness explanations by including them in the input.

Original: We ﬁnd that doing so improves GPT-3’s few-shotperformance by 18.96%, suggesting that explanations do aid models and are currently underutilizedin the CoT framework.
Revised: We ﬁnd that doing so improves GPT-3’s few-shotperformance by 18.96%, suggesting that explanations do aid models and are currently underutilizedin the CoT framework.

Original: Further analysis shows that, like humans, language models beneﬁt fromexplanations to learn with less data: UniﬁedQA with CoT obtains the same results as UniﬁedQAwithout CoT with only 40% of the training data.
Revised: Further analysis shows that, like humans, language models beneﬁt fromexplanations to learn with less data: UniﬁedQA with CoT obtains the same results as UniﬁedQAwithout CoT with only 40% of the training data.

Original: To sum up, our contributions are three-fold: (a) To bridge the gap in existing datasets in the scientiﬁcdomain, we build Science Question Answering ( SQA ), a new dataset containing 21,208 multimodalscience questions with rich domain diversity.
Revised: To sum up, our contributions are three-fold: (a) To bridge the gap in existing datasets in the scientiﬁcdomain, we build Science Question Answering ( SQA ), a new dataset containing 21,208 multimodalscience questions with rich domain diversity.

Original: To the best of our knowledge, SQA is the ﬁrst largescale multimodal dataset that annotates lectures and explanations for the answers.
Revised: To the best of our knowledge, SQA is the ﬁrst largescale multimodal dataset that annotates lectures and explanations for the answers.

Original: (b) We show that
Revised: (b) We show that

Original: CoT beneﬁts large language models in both few-shot and ﬁnetuning learning by improving modelperformance and reliability via generating explanations
Revised: CoT beneﬁts large language models in both few-shot and ﬁnetuning learning by improving modelperformance and reliability via generating explanations

Original: (c)
Revised: (c)

Original: We further explore the upper bound of
Revised: We further explore the upper bound of

Original: GPT-3 and show that CoT helps language models learn from fewer data.
Revised: GPT-3 and show that CoT helps language models learn from fewer data.

Original:  Related Work
Revised:  Related Work

Original: Visual question answering.
Revised: Visual question answering.

Original: Since the task of visual question answering (VQA) was ﬁrst proposedin [2], there have been plenty of VQA datasets [50, 52, 22, 10, 14, 11] conducted to facilitate theresearch work.
Revised: Since the task of visual question answering (VQA) was ﬁrst proposedin [2], there have been plenty of VQA datasets [50, 52, 22, 10, 14, 11] conducted to facilitate theresearch work.

Original: Although our SQA dataset shares some features with VQA, there are several maindifferences between them.
Revised: Although our SQA dataset shares some features with VQA, there are several maindifferences between them.

Original: First, SQA is more challenging than existing VQA datasets because it contains multimodal contexts and diverse topics in the scientiﬁc domain.
Revised: First, SQA is more challenging than existing VQA datasets because it contains multimodal contexts and diverse topics in the scientiﬁc domain.

Original: In addition, most answersare annotated with lectures and explanations, which makes SQA a suitable dataset for multi-modalquestion answering and multi-hop reasoning for AI systems.
Revised: In addition, most answersare annotated with lectures and explanations, which makes SQA a suitable dataset for multi-modelquestion answering and multi-hop reasoning for AI systems.

Original: Inspired by the recent remarkableperformance achieved for VQA [9, 24, 8], in this paper, we further extensively benchmark SQA witha wide range of attention-based [1, 30, 20, 9] and Transformer-based [28, 24, 25, 8] methods.
Revised: Inspired by the recent remarkableperformance achieved for VQA [9, 24, 8], in this paper, we further extensively benchmark SQA witha wide range of attention-based [1, 30, 20, 9] and Transformer-based [28, 24, 25, 8] methods.

Original: Datasets for science problems.
Revised: Datasets for science problems.

Original: Science problem solving is a challenging task that requires an AIsystem not only to understand the multimodal information from the science curriculum but also toreason about how to answer the domain-speciﬁc questions.
Revised: Science problem solving is a challenging task that requires an AIsystem not only to understand the multimodal information from the science curriculum but also toreason about how to answer the domain-speciﬁc questions.

Original: Current science problem datasets suchas AI2D [16], DVQA [15], VLQA [46], and FOODWEDS [23] have contributed to multimodalreasoning in the scientiﬁc domain.
Revised: Current science problem datasets suchas AI2D [16], DVQA [15], VLQA [46], and FOODWEDS [23] have contributed to multimodalreasoning in the scientiﬁc domain.

Original: These datasets, however, lack annotated explanations for theanswers to reveal the reasoning steps.
Revised: These datasets, however, lack annotated explanations for theanswers to reveal the reasoning steps.

Original: Some other datasets annotate the answers in the formsof supporting facts [33, 19], entailment trees [7], explanation graphs [12], reasoning chains [13].
Revised: Some other datasets annotate the answers in the formsof supporting facts [33, 19], entailment trees [7], explanation graphs [12], reasoning chains [13].

Original: However, these datasets are restricted to the single text modality with small data scales and limitedtopics.
Revised: However, these datasets are restricted to the single text modality with small data scales and limitedtopics.

Original: Instead, our SQA annotates the answers with grounded lectures and explanations.
Revised: Instead, our SQA annotates the answers with grounded lectures and explanations.

Original: Besides,
Revised: Besides,

Original: SQA features a richer domain diversity across 3 subjects, 26 topics, 127 categories, and 379 skills.
Revised: SQA features a richer domain diversity across 3 subjects, 26 topics, 127 categories, and 379 skills.

Original: Learning from explanations and few-shot Learning.
Revised: Learning from explanations and few-shot Learning.

Original: Explanations help humans understand a taskbetter, and there have been several attempts to show the same for models.
Revised: Explanations help humans understand a taskbetter, and there have been several attempts to show the same for models.

Original: For examples, the learningfrom instruction paradigm [35, 38, 47, 34] where the task level explanation is provided in the form ofinstruction improves model performance signiﬁcantly.
Revised: For examples, the learningfrom instruction paradigm [35, 38, 47, 34] where the task level explanation is provided in the form ofinstruction to improve model performance signiﬁcantly.

Original: An example of learning from explanationsin the scientiﬁc domain is proposed in [45] where the model interprets demonstrative solutions tosolve geometry problems.
Revised: An example of learning from explanationsin the scientiﬁc domain is proposed in [45] where the model interprets demonstrative solutions tosolve geometry problems.

Original: Recently, there has been a surge of interest in few-shot learning, wherelanguage models learn a speciﬁc task from a few examples [40, 3].
Revised: Recently, there has been a surge of interest in few-shot learning, wherelanguage models learn a speciﬁc task from a few examples [40, 3].

Original: For instance, [37, 48] ﬁnd thatexplanations in the format of the chain of thought can improve the reasoning ability of languagemodels in few-shot learning.
Revised: For instance, [37, 48] ﬁnd thatexplanations in the format of the chain of thought can improve the reasoning ability of languagemodels in few-shot learning.

Original: In this paper, we show that the chain of thought boosts the performanceof large language models like UniﬁedQA [18] if the models generate explanations along with theanswer in a ﬁne-tuned way.
Revised: In this paper, we show that the chain of thought boosts the performanceof large language models like UniﬁedQA [18] if the models generate explanations along with theanswer in a ﬁne-tuned way.

Original: Furthermore, a few-shot GPT-3 model via chain-of-thought prompting isable to improve the reasoning performance on SQA and generate reasonable explanations.
Revised: Furthermore, a few-shot GPT-3 model via chain-of-thought prompting isable to improve the reasoning performance on SQA and generate reasonable explanations.

Original:  Dataset
Revised:  Dataset

Original: We collect SQA , which is a multimodal multiple-choice science question dataset containing 21,208examples.
Revised: We collect SQA , which is a multimodal multiple-choice science question dataset containing 21,208examples.

Original: An example in SQA is shown in Figure 1.
Revised: An example in SQA is shown in Figure 1.

Original: Given the science question and multimodalcontexts, the task is to select the correct answer from multiple options.
Revised: Given the science question and multimodalcontexts, the task is to select the correct answer from multiple options.

Original: Different from existingdatasets [44, 16, 46, 29, 23], SQA covers diverse topics across three subjects: natural science, socialscience, and language science.
Revised: Different from existingdatasets [44, 16, 46, 29, 23], SQA covers diverse topics across three subjects: natural science, socialscience, and language science.

Original: Moreover, most questions are annotated with grounded lecturesand detailed explanations.
Revised: Moreover, most questions are annotated with grounded lecturesand detailed explanations.

Original: The lecture provides general knowledge that introduces the backgroundinformation for solving problems of a similar class.
Revised: The lecture provides general knowledge that introduces the backgroundinformation for solving problems of a similar class.

Original: The explanation reveals a speciﬁc reason forthe answer.
Revised: The explanation reveals a speciﬁc reason forthe answer.

Original: To effectively answer the questions, a model often needs to be able to understand themultimodal content in the input and extract external knowledge, similar to how humans do.
Revised: To effectively answer the questions, a model often needs to be able to understand themultimodal content in the input and extract external knowledge, similar to how humans do.

Original: Moreimportantly, the goal of SQA is to aid development of a reliable model that is capable of generatinga coherent chain of thought when arriving at the correct answer to reveal the multi-step reasoningprocess.
Revised: Moreimportantly, the goal of SQA is to aid development of a reliable model that is capable of generatinga coherent chain of thought when arriving at the correct answer to reveal the multi-step reasoningprocess.

Original: For data collection details, see Appendix A.1.
Revised: For data collection details, see Appendix A.1.

Original: [Table]

Original: Statistic

Original: [Table]
Revised: [Table]

Original: 3.1 Comparisons with Existing Datasets
Revised: 3.1 Comparisons with Existing Datasets

Original: Table 1 shows a comparison of SQA and other science problem datasets.
Revised: Table 1 shows a comparison of SQA and other science problem datasets.

Original: As shown in the table,
Revised: As shown in the table,

Original: SQA is much larger than most other datasets.
Revised: SQA is much larger than most other datasets.

Original: SQA also has the largest set of images, spans acrossall 12 grades, contains the longest questions, and has the most diverse input sources.
Revised: SQA also has the largest set of images, spans across [Table] all 12 grades, contains the longest questions, and has the most diverse input sources.

Original: As opposed tolimiting the subject to only natural science, SQA also includes social science and language science,largely adding to the domain diversity of the dataset.
Revised: As opposed tolimiting the subject to only natural science, SQA also includes social science and language science,largely adding to the domain diversity of the dataset.

Original: Furthermore, most of the questions in SQA areannotated with textual lectures (83.9%) and explanations (90.5%), which reveal the reasoning path tothe correct answer.
Revised: Furthermore, most of the questions in SQA areannotated with textual lectures (83.9%) and explanations (90.5%), which reveal the reasoning path tothe correct answer.

Original: To the best of our knowledge, SQA is the ﬁrst large-scale multimodal sciencequestion dataset that annotates the answers with detailed lectures and explanations.
Revised: To the best of our knowledge, SQA is the ﬁrst large-scale multimodal sciencequestion dataset that annotates the answers with detailed lectures and explanations.

Original: 3.2 Data Analysis
Revised: 3.2 Data Analysis

Original: Key statistics.
Revised: Key statistics.

Original: We randomly split the dataset into training, validation, and test splits with a ratio of60:20:20.
Revised: We randomly split the dataset into training, validation, and test splits with a ratio of60:20:20.

Original: Each split has 12,726, 4,241, and 4,241 examples, respectively.
Revised: Each split has 12,726, 4,241, and 4,241 examples, respectively.

Original: Table 2 shows the mainstatistics of SQA .
Revised: Table 2 shows the mainstatistics of SQA .

Original: SQA has a large set of different questions, totaling up to 9,122.
Revised: SQA has a large set of unique questions, totaling up to 9,122.

Original: Out of the 21,208questions in SQA , 10,332 (48.7%) have an image context, 10,220 (48.2%) have a text context, and6,532 (30.8%) have both.
Revised: Out of the 21,208questions in SQA , 10,332 (48.7%) have an image context, 10,220 (48.2%) have a text context, and6,532 (30.8%) have both.

Original: 83.9% of the questions are annotated with a lecture, while 91.3% of thequestions feature an explanation.
Revised: 83.9% of the questions are annotated with a lecture, while 91.3% of thequestions feature an explanation.

Original: The cross-combination of these information sources diversiﬁes theproblem scenario: sometimes the model is given a lot of information from multiple sources, while atother times, the only source of information is the question itself.
Revised: The cross-combination of these information sources diversiﬁes the [Table] problem scenario: sometimes the model is given a lot of information from multiple sources, while at [Table]

Original:  This level of complexity is verycommon in grade-level science exams.
Revised: other times, the only source of information is the question itself. This level of complexity is very

Original: [Figure]
Revised: [Table]

Original: [Figure] [Figure] [Figure]
Revised: grade

Original: [Figure]
Revised: [Table]

Revised: science

Original: [Figure]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: Question analysis.
Revised: Question analysis.

Original: SQA has a diverse set of science questions.
Revised: SQA has a diverse set of science questions.

Original: Figure 2 shows a distribution ofthe ﬁrst four words in the question text.
Revised: Figure 2 shows a distribution ofthe ﬁrst four words in the question text.

Original: A large number of question lengths and formats highlightthe diversity of SQA .
Revised: A large number of question lengths and formats highlightthe diversity of SQA .

Original: The question lengths range from 3 words to 141 words, and the questions in
Revised: The question lengths range from 3 words to 141 words, and the questions in

Original: SQA have an average length of 12.11 words.
Revised: SQA have an average length of 12.11 words.

Original: The question length distribution is visualized againstother VQA datasets in Figure 3 (a).
Revised: The question length distribution is visualized against [Table] other VQA datasets in Figure 3 (a).

Original: As shown in the diagram, SQA ’s distribution is ﬂatter than otherdatasets, spanning more evenly across different question lengths.
Revised: As shown in the diagram, SQA ’s distribution is ﬂatter than otherdatasets, spanning more evenly across different question lengths.

Original: Context analysis.
Revised: Context analysis.

Original: Figure 3 (b) shows the number and percentage of questions with either an imagecontext, a text context, or both.
Revised: Figure 3 (b) shows the number and percentage of questions with either an imagecontext, a text context, or both.

Original: There are a total of 7,803 unique image contexts and 4,651 unique textcontexts.
Revised: There are a total of 7,803 unique image contexts and 4,651 unique textcontexts.

Original: 66.11% of the questions have at least one type of context information.
Revised: 66.11% of the questions have at least one type of context information.

Original: The image contextis in the format of diagrams or natural images, which visualize the critical scenario necessary forquestion answering or simply illustrate the question for better understanding.
Revised: The image contextis in the format of diagrams or natural images, which visualize the critical scenario necessary forquestion answering or simply illustrate the question for better understanding.

Original: Similarly, the textualcontext can provide either semantically rich information or a simple hint to the question.
Revised: Similarly, the textualcontext can provide either semantically rich information or a simple hint to the question.

Original: Therefore,models need to be ﬂexible and general to understand these diverse types of contexts.
Revised: Therefore,models need to be ﬂexible and general to understand these diverse types of contexts.

Original: Domain analysis.
Revised: Domain analysis.

Original: Each SQA question belongs to one of the three subjects: natural science, languagescience, and social science.
Revised: Each SQA question belongs to one of the three subjects: natural science, languagescience, and social science.

Original: With each subject, they are categorized ﬁrst by the topic ( Biology , Physics ,
Revised: With each subject, they are categorized ﬁrst by the topic ( Biology , Physics ,

Original: Chemistry , etc.), then by the category ( Plants , Cells , Animals , etc.), and ﬁnally by the speciﬁc skill( Classify fruits and vegetables as plant parts , Identify countries of Africa , etc.).
Revised: Chemistry , etc.), then by the category ( Plants , Cells , Animals , etc.), and ﬁnally by the speciﬁc skill( Classify fruits and vegetables as plant parts , Identify countries of Africa , etc.).

Original: SQA has a total of 26topics, 127 categories, and 379 skills.
Revised: SQA has a total of 26topics, 127 categories, and 379 skills.

Original: The treemap in Figure 9 visualizes the different subjects, topics,and categories and shows that SQA questions are very diverse, spanning a wide range of domains.
Revised: The treemap in Figure 4 visualizes the different subjects, topics,and categories and shows that SQA questions are very diverse, spanning a wide range of domains.

Original:  Baselines and Chain-of-Thought Models
Revised:  Baselines and Chain-of-Thought Models

Original: In this section, we establish various baselines and develop two chain-of-thought models on SQA .
Revised: In this section, we establish various baselines and develop two chain-of-thought models on SQA .

Original: 4.1 Baselines
Revised: 4.1 Baselines

Original: Heuristic baselines.
Revised: Heuristic baselines.

Original: The ﬁrst heuristic baseline is random chance : we randomly select one from themultiple options.
Revised: The ﬁrst heuristic baseline is random chance : we randomly select one from themultiple options.

Original: Each trial is completed on the whole test set, and we take three different trials foran average result.
Revised: Each trial is completed on the whole test set, and we take three different trials foran average result.

Original: The second heuristic baseline is human performance .
Revised: The second heuristic baseline is human performance .

Original: We post the task to Amazon
Revised: We post the task to Amazon

Original: Mechanical Turk and ask workers to answer SQA questions.
Revised: Mechanical Turk and ask workers to answer SQA questions.

Original: Only workers who obtain a high schoolor higher degree and pass the qualiﬁcation examples are qualiﬁed for the study.
Revised: Only workers who obtain a high schoolor higher degree and pass the qualiﬁcation examples are qualiﬁed for the study.

Original: Each worker needs toanswer a set of 10 test questions, and each question is answered by three different workers.
Revised: Each worker needs toanswer a set of 10 test questions, and each question is answered by three different workers.

Original: For moredetails of the human performance study, see Appendix B.2.
Revised: For moredetails of the human performance study, see Appendix B.2.

Original: Zero-shot and few-shot baselines.
Revised: Zero-shot and few-shot baselines.

Original: We establish the zero-shot baselines on top of UniﬁedQA[18] and GPT-3 [5].
Revised: We establish the zero-shot baselines on top of UniﬁedQA[18] and GPT-3 [5].

Original: The zero-shot setup follows the format of QCM !
Revised: The zero-shot setup follows the format of QCM !

Original: A where the input is theconcatenation of tokens of the question text (Q), the context text (C), and multiple options (M), whilethe output is to predict the answer (A) from the option set.
Revised: A where the input is theconcatenation of tokens of the question text (Q), the context text (C), and multiple options (M), whilethe output is to predict the answer (A) from the option set.

Original: We extract the caption from the captioningmodel based on ViT [8] and GPT-2 [41] for the image as the visual context.
Revised: We extract the caption from the captioningmodel based on ViT [8] and GPT-2 [41] for the image as the visual context.

Original: In the few-shot setting, wefollow the standard prompting [4] where in-context examples from the training set are concatenatedbefore the test instance.
Revised: In the few-shot setting, wefollow the standard prompting [4] where in-context examples from the training set are concatenatedbefore the test instance.

Original: These in-context examples serve as an instruction for the language model toadjust to the speciﬁc task in SQA .
Revised: These in-context examples serve as an instruction for the language model toadjust to the speciﬁc task in SQA .

Original: Fine-tuning baselines.
Revised: Fine-tuning baselines.

Original: We ﬁrst consider the ﬁne-tuning baselines from VQA models [1, 20, 49, 9,21, 31, 24] proposed in recent years.
Revised: We ﬁrst consider the ﬁne-tuning baselines from VQA models [1, 20, 49, 9,21, 31, 24] proposed in recent years.

Original: These VQA baselines take the question, the context, and choicesas the textual input, take the image as the visual input, and predict the score distribution over choicecandidates via a linear classiﬁer.
Revised: These VQA baselines take the question, the context, and choicesas the textual input, take the image as the visual input, and predict the score distribution over choicecandidates via a linear classiﬁer.

Original: In addition, we build the ﬁne-tuning baseline on top of the largelanguage model UniﬁedQA [18].
Revised: In addition, we build the ﬁne-tuning baseline on top of the largelanguage model UniﬁedQA [18].

Original: UniﬁedQA takes the textual information as the input and outputsthe answer option.
Revised: UniﬁedQA takes the textual information as the input and outputsthe answer option.

Original: Similarly, the image is converted into a caption that provides the visual semanticsfor the language model.
Revised: Similarly, the image is converted into a caption that provides the visual semanticsfor the language model.

Original: 4.2 Language Models with the Chain of Thought
Revised: 4.2 Language Models with the Chain of Thought

Original: A chain of thought refers to a coherent ﬂow of sentences that reveals the premises and conclusion ofa reasoning problem [48].
Revised: A chain of thought refers to a coherent ﬂow of sentences that reveals the premises and conclusion ofa reasoning problem [48].

Original: A chain of thought clearly decomposes a multi-hop reasoning task intointermediate steps instead of solving the task in a black-box way.
Revised: A chain of thought clearly decomposes a multi-hop reasoning task intointermediate steps instead of solving the task in a black-box way.

Original: The chain of thought can be thestep-by-step thought process [48] before arriving at the ﬁnal answer or explanations [36] that comeafter the answer.
Revised: The chain of thought can be thestep-by-step thought process [48] before arriving at the ﬁnal answer or explanations [36] that comeafter the answer.

Original: The annotated lectures and explanations in SQA serve as demonstrations of the chain of thought that mimics the multi-step reasoning steps of human beings.
Revised: The annotated lectures and explanations in SQA serve as demonstrations of thechain of thought that mimics the multi-step reasoning steps of human beings.

Original: In this paper, we studyif large language models can generate reasonable explanations as the chain of thought to reveal thethought process when answering SQA questions.
Revised: In this paper, we studyif large language models can generate reasonable explanations as the chain of thought to reveal thethought process when answering SQA questions.

Original: Further, we explore how the chain of thought canimprove the reasoning ability of language models on SQA in both few-shot and ﬁne-tuning learning.
Revised: Further, we explore how the chain of thought canimprove the reasoning ability of language models on SQA in both few-shot and ﬁne-tuning learning.

Original: UniﬁedQA with the chain of thought.
Revised: UniﬁedQA with the chain of thought.

Original: UniﬁedQA [18] is a state of the art model for multi-optionquestion answering.
Revised: UniﬁedQA [18] is a state of the art model for multi-optionquestion answering.

Original: The original architecture of UniﬁedQA takes the question and options as theinput and outputs a short phrase as the ﬁnal answer.
Revised: The original architecture of UniﬁedQA takes the question and options as theinput and outputs a short phrase as the ﬁnal answer.

Original: We make a format modiﬁcation to develop
Revised: We make a format modiﬁcation to develop

Original: UniﬁedQA with the chain of thought (CoT) i.e. UniﬁedQA is ﬁne-tuned to generate a long sequenceof text which consists of the answer followed by the lecture and explanation.
Revised: UniﬁedQA with the chain of thought (CoT) i.e. UniﬁedQA is ﬁne-tuned to generate a long sequenceof text which consists of the answer followed by the lecture and explanation.

Original: GPT-3 via chain-of-thought prompting.
Revised: GPT-3 via chain-of-thought prompting.

Original: Recent research work [5] has shown that GPT-3 [5] canperform various tasks when provided in-context examples in a standard prompt.
Revised: Recent research work [5] has shown that GPT-3 [5] canperform various tasks when provided in-context examples in a standard prompt.

Original: Take multi-optionquestion answering as an example, the standard prompt [32, 51, 27] builds instructions using incontext examples with components of the question text, options, and the correct answer text.
Revised: Take multi-optionquestion answering as an example, the standard prompt [32, 51, 27] builds instructions using incontext examples with components of the question text, options, and the correct answer text.

Original: This styleof few-shot learning enables the GPT-3 model to answer speciﬁc questions without parameter updates.
Revised: This styleof few-shot learning enables the GPT-3 model to answer speciﬁc questions without parameter updates.

Original: Different from standard prompting, we build GPT-3 via chain-of-thought (CoT) prompting, as shownin Figure 4.
Revised: Different from standard prompting, we build GPT-3 via chain-of-thought (CoT) prompting, as shownin Figure 5.

Original: To be speciﬁc, for each test problem t , we map the prompt instruction I : { I i } n , I t intoa textual format where { I i } n refers to the instruction set of n -shot in-context examples from thetraining set, while I t denotes the test instruction.
Revised: To be speciﬁc, for each test problem t , we map the prompt instruction I : { I i } n , I t intoa textual format where { I i } n refers to the instruction set of n -shot in-context examples from thetraining set, while I t denotes the test instruction.

Original: Instead of the way where the explanation comesbefore the answer [48], we feed the instruction I into the encoder-decoder model GPT-3 to generatethe answer a followed by the lecture lect and explanation exp : M : { I i } n , I t !
Revised: Instead of the way where the explanation comesbefore the answer [48], we feed the instruction I into the encoder-decoder model GPT-3 to generatethe answer a followed by the lecture lect and explanation exp : M : { I i } n , I t !

Original: a, lect, exp .
Revised: a, lect, exp .

Original: Question: question :

Original: I

Original: [Figure]
Revised: [Figure]

Original:  Experiments
Revised:  Experiments

Original: [Figure]

Original: 5.1 Experimental Setup
Revised: 5.1 Experimental Setup

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: The heuristics and VQA baselines treat our SQA task as a multi-class classiﬁcation problem with multiple options and are evaluated with the accuracy metrics.
Revised: The heuristics and VQA baselines treat our SQA task as a multi-class classiﬁcation problem with multiple options and are evaluated with the accuracy metrics.

Original: UniﬁedQA and
Revised: UniﬁedQA and

Original: GPT-3 treat SQA as a text generation problem.
Revised: GPT-3 treat SQA as a text generation problem.

Original: So the most similar option is selected as the ﬁnalprediction to evaluate the question answering accuracy.
Revised: So the most similar option is selected as the ﬁnalprediction to evaluate the question answering accuracy.

Original: The generated lectures and explanations areevaluated by automatic metrics [39, 26, 43] and human scores by annotators. 
Revised: The generated lectures and explanations areevaluated by automatic metrics [39, 26, 43] and human scores by annotators. [Table]

Original: Implementation details.
Revised: Implementation details.

Original: The VQA baselines are trained for a maximum number of 50 epochs with alearning rate of 5 e  5 .
Revised: The VQA baselines are trained for a maximum number of 50 epochs with alearning rate of 5 e  5 .

Original: We ﬁne-tune the UniﬁedQA for 50 k iterations and evaluate every 1 k iteration.
Revised: We ﬁne-tune the UniﬁedQA for 50 k iterations and evaluate every 1 k iteration.

Original: The training process is stopped following the early stopping strategy with a patience period of threeevaluations.
Revised: The training process will be stopped following the early stopping strategy with a patience period ofthree evaluations.

Original: For GPT-3, we use the text-davinci-002 engine, which is the most capable modelversion suggested in the ofﬁcial documentation.
Revised: For GPT-3, we use the text-davinci-002 engine, which is the most capablemodel version suggested in the ofﬁcial documentation.

Original: More details can be found in Appendix B.1.
Revised: More details can be found in Appendix B.1.

Original: 5.2 Results for Question Answering
Revised: 5.2 Results for Question Answering

Original: Table 3 demonstrates the empirical results for Science Question Answering.
Revised: Table 3 demonstrates the empirical results for Science Question Answering.

Original: VQA baselines.
Revised: VQA baselines.

Original: We feed the VQA baseline models with the input of QCM format to predict answers
Revised: We feed the VQA baseline models with the input of QCM format to predict answers

Original: A. Out of all the VQA models we benchmarked, VisualBERT [24, 25] performs the best on average(61.87%).
Revised: A. Out of all the VQA models we benchmarked, VisualBERT [24, 25] performs the best on average(61.87%).

Original: Interestingly, Patch-TRM [31] beats VisualBERT in natural science (NAT) and languagescience (LAN), and it also performs better in higher-grade questions (67.50% v.s. 59.92%).
Revised: Interestingly, Patch-TRM [31] beats VisualBERT in natural science (NAT) and languagescience (LAN), and it also performs better in higher-grade questions (67.50% v.s. 59.92%).

Original: However,in the subject of social science (SOC), VisualBERT outperforms Patch-TRM by a large margin [Table] (+22.39%).
Revised: However,in the subject of social science (SOC), VisualBERT outperforms Patch-TRM by a large margin(+22.39%).

Original: Such drastic changes in performance might imply that current VQA models are notgeneralized to process the challenging questions in SQA .
Revised: Such drastic changes in performance might imply that current VQA models are notgeneralized to process the challenging questions in SQA .

Original: Language models.
Revised: Language models.

Original: We evaluate whether large-scale pretraining on text can help language modelslearn scientiﬁc knowledge and thus perform better on the SQA task.
Revised: We evaluate whether large-scale pretraining on text can help language modelslearn scientiﬁc knowledge and thus perform better on the SQA task.

Original: For this purpose, we have triedtwo of the state-of-the-art pre-trained language models: UniﬁedQA and GPT-3.(i)
Revised: For this purpose, we have triedtwo of the state-of-the-art pre-trained language models: UniﬁedQA and GPT-3.(i)

Original: UniﬁedQA.
Revised: UniﬁedQA.

Original: The results show that without any supervised ﬁne-tuning (zero-shot), UniﬁedQAcannot beat any VQA baseline model, while the pretraining does help the model obtain some scientiﬁcknowledge to outperform the random baseline.
Revised: The results show that without any supervised ﬁne-tuning (zero-shot), UniﬁedQAcannot beat any VQA baseline model, while the pretraining does help the model obtain some scientiﬁcknowledge to outperform the random baseline.

Original: By training the T5 model (UniﬁedQA backbone) withthe answer labels in SQA , UniﬁedQA BASE reports an accuracy of 70.12% on average.
Revised: By training the T5 model (UniﬁedQA backbone) withthe answer labels in SQA , UniﬁedQA BASE reports an accuracy of 70.12% on average.

Original: By furtherteaching the model to generate the answer along with lecture and explanation, the developed languagemodel with chain-of-thought (UniﬁedQA BASE (CoT)) brings additional improvements of +3.21%(QCM !
Revised: By furtherteaching the model to generate the answer along with lecture and explanation, the developed language [Table] model with chain-of-thought (UniﬁedQA BASE (CoT)) brings additional improvements of +3.21%(QCM !

Original: AE) and +3.99% (QCM !
Revised: AE) and +3.99% (QCM !

Original: ALE).
Revised: ALE).

Original: These results show that generating the chain of thoughtalong with the answer beneﬁts the reasoning ability of language models.(ii) GPT-3.
Revised: These results show that generating the chain of thoughtalong with the answer beneﬁts the reasoning ability of language models.(ii) GPT-3.

Original: The positive effect of pretraining is also proved by the surprisingly good results from
Revised: The positive effect of pretraining is also proved by the surprisingly good results from

Original: GPT-3 in the same zero-shot setting as UniﬁedQA.
Revised: GPT-3 in the same zero-shot setting as UniﬁedQA.

Original: Without any ﬁne-tuning, GPT-3 already reachesalmost the best performance we can get.
Revised: Without any ﬁne-tuning, GPT-3 already reachesalmost the best performance we can get.

Original: Interestingly, prompting the GPT-3 with two trainingexamples with just answers results in a negligible difference.
Revised: Interestingly, prompting the GPT-3 with two trainingexamples with just answers results in a negligible difference.

Original: However, if we prompt GPT-3 withchain-of-thought prompting (QCM !
Revised: However, if we prompt GPT-3 withchain-of-thought prompting (QCM !

Original: ALE), we obtain the state-of-the-art results  so far (75.17%).
Revised: ALE), we obtain the state-of-the-art results on SQA so far(75.17%) on SQA .

Original: Human performance.
Revised: Human performance.

Original: Humans outperform all benchmarks consistently across question classes,context types, and grades, e.g., a 20.07% gap for questions with the image context (IMG) betweenhumans and our best performing model.
Revised: Humans outperform all benchmarks consistently across question classes,context types, and grades, e.g., a 20.07% gap for questions with the image context (IMG) betweenhumans and our best performing model.

Original: The gap is to be ﬁlled by future research on multimodalreasoning for scientiﬁc question answering.
Revised: The gap is to be ﬁlled by future research on multimodalreasoning for scientiﬁc question answering.

Original: [Table]

Original: 5.3 Results for Generated Explanations
Revised: 5.3 Results for Generated Explanations

Original: One prediction example of GPT-3 (CoT) is visualized in Figure 5.
Revised: One prediction example of GPT-3 (CoT) is visualized in Figure 6.

Original: We can see that GPT-3 (CoT)predicts the correct answer and generates a reasonable lecture and explanation to mimic the humanthought process.
Revised: We can see that GPT-3 (CoT)predicts the correct answer and generates a reasonable lecture and explanation to mimic the humanthought process.

Original: We further report automatic metrics (BLEU-1/4 [39], ROUGE-L [39], and (sentence)
Revised: We further report automatic metrics (BLEU-1 [39], ROUGE-L [39], and (sentence)

Original: Similarity [43]) to evaluate the generated lectures and explanations, as shown in Table 4.
Revised: Similarity [43]) to evaluate the generated lectures and explanations, as shown in Table 4.

Original: The
Revised: The

Original: Similarity metric computes the cosine-similarity of semantic embeddings between two sentencesbased on the Sentence-BERT network [43].
Revised: Similarity metric computes the cosine-similarity of semantic embeddings between two sentencesbased on the Sentence-BERT network [43].

Original: The results show that UniﬁedQA BASE (CoT) generates themost similar explanations to the given ones.
Revised: The results show that UniﬁedQA BASE (CoT) generates themost similar explanations to the given ones.

Original: However, it’s commonly agreed that automatic evaluationof generated texts only provides a partial view and has to be complemented by a human study.
Revised: However, it’s commonly agreed that automatic evaluationof generated texts only provides a partial view and has to be complemented by a human study.

Original: Byasking annotators to rate the relevance, correctness, and completeness of generated explanations, weﬁnd that the explanations generated by GPT-3 (CoT) conform best to human judgment.[Figure]
Revised: Byasking annotators to rate the relevance, correctness, and completeness of generated explanations, weﬁnd that the explanations generated by GPT-3 (CoT) conform best to human judgment.

Original: [Table]
Revised: [Table]

Original: 5.4 Analysis
Revised: 5.4 Analysis

Original: Blind studies.
Revised: Blind studies.

Original: Blind studies are conducted on top of the modiﬁcation of the full model, Top-Down[1].
Revised: Blind studies are conducted on top of the modiﬁcation of the full model, Top-Down[1].

Original: The results achieved in blind studies of Q only and C I only are close to random chance, showingthat the SQA dataset is robust and reliable in distribution.
Revised: The results achieved in blind studies of Q only and C I only are close to random chance, showingthat the SQA dataset is robust and reliable in distribution.

Original: The performance drops in Q+M only,
Revised: The performance drops in Q+M only,

Original: Q+C T +M only, and Q+C I +M only indicate that all input components provide critical information foranswering SQA questions.
Revised: Q+C T +M only, and Q+C I +M only indicate that all input components provide critical information foranswering SQA questions.

Original: Prompt types.
Revised: Prompt types.

Original: We study the effect of prompt types and visualize the comparison in Figure 6 (a).
Revised: We study the effect of prompt types and visualize the comparison in Figure 7 (a).

Original: It shows that prompting the GPT-3 model with both lecture and explanation (QCM !
Revised: It shows that prompting the GPT-3 model with both lecture and explanation (QCM !

Original: ALE) resultsin the highest accuracy on average and the smallest variance.
Revised: ALE) resultsin the highest accuracy on average and the smallest variance.

Original: In contrast, prompting with only theexplanation (QCM ! AE) gives the largest variance, resulting in a less stable model.
Revised: In contrast, prompting with only theexplanation (QCM ! AE) gives the largest variance, resulting in a less stable model.

Original: [Figure]
Revised: [Figure]

Original: Number of in-context examples.
Revised: Number of in-context examples.

Original: In Figure 6 (b), we further investigate how different numbers oftraining examples encoded in prompts can affect the prediction accuracy.
Revised: In Figure 7 (b), we further investigate how different numbers oftraining examples encoded in prompts can affect the prediction accuracy.

Original: The QCM !
Revised: The QCM !

Original: ALE prompt type outperforms or performs comparably the QCM !
Revised: ALE prompttype outperforms or performs comparably the QCM !

Original: A type with all numbers of examples.
Revised: A type with all numbers of examples.

Original: And weobserve the peak performance of QCM !
Revised: And weobserve the peak performance of QCM !

Original: ALE with 2 training examples being prompted.
Revised: ALE with 2 training examples being prompted.

Original: After that,the accuracy goes down as more training examples are added to the model.
Revised: After that,the accuracy goes down as more training examples are added to the model.

Original: [Table]
Revised: [Table]

Original: Dynamic sampling.
Revised: Dynamic sampling.

Original: In Table 5, instead of random sampling, we try to dynamically select thein-context examples to prompt with the sameclass as the test sample.
Revised: In Table 5, instead of random sampling, we try to dynamically select thein-context examples to prompt with the sameclass as the test sample.

Original: However, slight differences in prediction accuracy are observed whencomparing them to simple random sampling. 
Revised: However, slight differences in prediction accuracy are observed whencomparing them to simple random sampling. [Table]

Original: Upper bound.
Revised: Upper bound.

Original: We search the upper bound of the GPT-3 accuracy by feeding the gold lecture andexplanation in the test prompt.
Revised: We search the upper bound of the GPT3 accuracy by feeding the gold lecture and explanationin the test prompt.

Original: As reported in Table 6, QCME* !
Revised: As reported in Table 6, QCME* !

Original: A outperforms the QCM !
Revised: Aoutperforms the QCM !

Original: ALEbaseline by 18.86% and QCMLE* !
Revised: ALE baseline by 18.86% and QCMLE* !

Original: A outperforms QCM !
Revised: A outperforms QCM !

Original: ALE by 18.96%, indicating a potentialimprovement direction by generating correct explanations before answering science questions.[Table] [Table]
Revised: ALE by 18.96%, indicating a potential improvement direction by generatingcorrect explanations before answering science questions.

Original: Positions of lectures and explanations.

Original: We study the performance of GPT-3 (CoT) in terms ofdifferent positions of lectures and explanations on 1,000 test examples.

Original: Results in Table 7 there couldbe huge accuracy decreases if GPT-3 (CoT) predicts the lectures and explanations before answers.

Original: Itis mainly because if GPT-3 (CoT) is formalized to generate the long lecture and explanation ﬁrst,there is a larger chance that it stops generating the prediction early or use up the maximum tokenlimits before obtaining the required answer.

Original: CoT learns with fewer data.
Revised: CoT learns with fewer data.

Original: To study if the chain ofthought helps language models learn more efﬁciently, wereport the accuracies of UniﬁedQA and UniﬁedQA (CoT)ﬁne-tuned on different sizes of the training set in Figure 7.
Revised: To study if the chain ofthought helps language models learn more efﬁciently, wereport the accuracies of UniﬁedQA and UniﬁedQA (CoT)ﬁne-tuned on different sizes of the training set in Figure 8.

Original: UniﬁedQA (CoT) beneﬁts the language models by learningthe coherent reasoning path when answering SQA questions,resulting in similar accuracy with fewer training examples.
Revised: UniﬁedQA (CoT) beneﬁts the language models by learningthe coherent reasoning path when answering SQA questions,resulting in similar accuracy with fewer training examples.

Original: Error analysis.
Revised: Error analysis.

Original: GPT-3 via chain-of-chain prompting obtains promising results but still fails to answer a wide rangeof challenging questions in SQA .
Revised: GPT-3 via chain-of-chain prompting obtains promising results but still fails to answer a wide rangeof challenging questions in SQA .

Original: See examples of failurecases in Appendix B.4.
Revised: See examples of failurecases in Appendix B.4.

Original: The failure cases can be classiﬁedinto two types: (a) the model fails to understand the multimodal inputs and lacks domain-speciﬁc knowledge to arrive at the correct answer; (b) the model generates the wrong chain of thought with irrelevant, incorrect, or incomplete information.
Revised: The failure cases can be classiﬁedinto two types: (a) the model fails to understand the multimodal inputs and lacks domain-speciﬁc knowledge to arrive at the correct answer; (b) the modelgenerates the wrong chain of thought with irrelevant, incorrect, or incomplete information.

Original:  Discussion and Conclusion
Revised:  Discussion and Conclusion

Original: In this paper, we propose Science Question Answering SQA , a dataset that features 21,208 multioption questions with multimodal contexts from the science curriculum.
Revised: In this paper, we propose Science Question Answering SQA , a dataset that features 21,208 multioption questions with multimodal contexts from the science curriculum.

Original: To the best of our knowledge,
Revised: To the best of our knowledge,

Original: SQA is the ﬁrst large-scale multimodal science dataset where most questions are annotated withcorresponding lectures and explanations.
Revised: SQA is the ﬁrst large-scale multimodal science dataset where most questions are annotated withcorresponding lectures and explanations.

Original: We establish various baselines, including recent VQAmodels and large language models on SQA .
Revised: We establish various baselines, including recent VQAmodels and large language models on SQA .

Original: We further study if language models can generatereasonable explanations and then beneﬁt the reasoning ability.
Revised: We further study if language models can generatereasonable explanations and then beneﬁt the reasoning ability.

Original: Experiments show that the UniﬁedQAwith the chain of thought can achieve an improvement of 3.99% and few-shot GPT-3 via chainof-thought (CoT) prompting 
Revised: Experiments show that the UniﬁedQAwith the chain of thought can achieve an improvement of 3.99% and few-shot GPT-3 via chainof-thought (CoT) prompting can obtain a satisfactory accuracy of 75.17% on SQA .

Revised: 65.2% of thegenerated explanations from GPT-3 (CoT) meet the gold standard by the human evaluations.