# A Survey on Machine Reading Comprehension--Tasks, Evaluation Metrics and Benchmark Datasets
## 2 Tasks
### Definition of Each Category in the New Classification Method
#### 2.4.2 Type of Questions  
According to the type of question, a MRC task can be classified into three categories: cloze style, natural form, and synthetic form:  
Figure 11: An example of textual MRC task. * Cloze style The cloze question is usually a sentence with a placeholder. Its sentence pattern may be a declarative sentence, an imperative sentence, and so forth, and is not necessarily an interrogative sentence. In addition, the sentence may also contain image information. The system is required to find a correct word, phrase or image that is suitable to be filled in the placeholder so that the sentence is complete. The cloze question can be defined as: Given the context \(C=\left\{c_{0},c_{1},\ldots c_{j},\ldots c_{j+n}\ldots,c_{lc}\right\}\)\(\left(0\leq j\leq lc,0\leq n\leq lc-1,c_{j}\in V\cup M\right)\), where \(l_{c}\) denotes the length of this context \(C\). \(A=\left\{c_{j},\ldots c_{j+n}\right\}\) is a short span in context \(C\). After replaced \(A\) with a placeholder \(X\), a cloze style question \(Q\) for context \(C\) is formed, it can be formulated as \(Q=\left\{c_{0},c_{1},\ldots X\ldots,c_{lc}\right\}\), in which the \(X\) is a placeholder. The answer to question \(Q\) is the \(A=\left\{c_{j},\ldots c_{j+n}\right\}\). According to the type of corpus, cloze questions also can be divided into textual and multi-modal. A textual cloze question is usually a sentence with a placeholder. The MRC system is required to find a correct word or phrase that is suitable to be filled in the placeholder so that the sentence is complete. An example of textual cloze question has been shown in Figure 7.  
A multi-modal cloze question is a natural sentence with visual information such as images, but some parts of these images are missing, and the MRC system is required to fill in the missing images. For example, a sample of visual cloze question in the RecipeQA [34] dataset is shown in Figure 12:  
* Natural form  
A question in natural form is a natural question that conforms to the grammar of natural language. Different from the cloze question, which contains placeholder, a natural form question is a complete sentence and a question that conforms to the grammatical rules. It could be defined as:  
**Definition 5**. : _In a MRC task, given a 'Natural' question \(Q\), it could be formulated as \(Q_{i}=\{q_{0},q_{1},\ldots q_{i},\ldots,q_{lq}\}\), where \(q_{i}\in V\cup M(0\leq i\leq lq)\). \(Q\) denotes a complete sentence (may also contain images) that conforms to the natural language grammar and \(l_{q}\) denotes the length of the question \(Q\)._  
In most cases, a 'Natural' question \(Q\) is an interrogative sentence that asks a direct question and is punctuated at the end with a question mark. However, in some cases, \(Q\) may not be an interrogative sentence but an imperative sentence, for example, \"please find the correct statement from the following options.\"  
In addition, according to the type of corpus, natural form questions can be divided into textual and multi-modal. Textual natural question is usually a natural question or imperative sentence. With some graphics or video, the multi-modal natural question is also a natural question or imperative. Example of textual natural question is shown in Figure 13 below, and example of multi-modal natural question has been shown in Figure 4.  
* Synthetic style  
The synthetic form of the question is just a list of words and do not necessarily conform to normal grammatical rules. Common datasets with synthetic form questions are Qangaroo, WikiReading, and so on. Take Qangaroo as an example, in the Qangaroo dataset, the question is replaced by a collection of attribute words. The 'question' here is not a complete sentence that fully conforms to the natural language grammar, but a combination of words. The synthetic form of the question can be defined as:  
**Definition 6**. : _In a MRC task, given a 'Synthetic style' question \(Q\), it could be formulated as \(Q_{i}=\{q_{0},q_{1},\ldots q_{j},\ldots q_{lq}\}\), where \(q_{i}\in V\cup M(0\leq i\leq lq)\).