## 4 Experiments
### Information Type Analysis
Some examples requiring different information are shown in Figure 3. The first item shows two passage-question cases requiring lexical knowledge. These two questions are easily answered by matching the similar words/phrases in the question and passage (\"before meals\" vs. \"empty stomach\" and \"can purchase\" vs. \"purchase\"). As we can see, our model tends to select the first block to answer this kind of question. The questions requiring syntactic knowledge are shown in the second item. In this kind of problem, the syntactic structures of the question and the evidence are usually different. \"Evidence\" is the most needed sentences in the passage to answer the question. For instance, in the first example, the question uses the active voice while the evidence \"Fever... be caused not by hernia\" uses the passive voice. To answer this question, the model needs to find the correct subject-predicate-object tuples in the question and evidence. The third item shows two examples that requires semantic knowledge. This kind of example usually requires understanding the semantic information of the passages at first, then answers the question with logical reasoning. Take the first question being the third item as an example, the model firstly understands \"detention\" is a worse outcome, then reasons that \"the defendant must participate in the cross-examination after receiving court summons\", and finally gives the answer \"no\". \begin{table}
\begin{tabular}{l c c c} \hline
**Information Type** & **1 st Layer** & **2 nd Layer** & **3 rd Layer** \\ \hline Lexical Knowledge & 51.0 & 32.0 & 10.0 \\ Syntactic Knowledge & 37.0 & 46.0 & 26.0 \\ Semantic Knowledge & 13.0 & 23.0 & 60.0 \\ Specific Knowledge & 1.0 & 4.0 & 10.0 \\ \hline \end{tabular}
\end{table} 
Table 5: The frequencies of the required information in each block on the ReCO dataset. Some questions may require more than one type of information. The last item is some cases requiring external knowledge, such as \"fish is a kind of seafood\". There are no specific rules for this kind of problem. Generally speaking, the more knowledge the model learns from the training corpus, the better the performance will be. Moreover, we also show the predicted results of a common BERT baseline in Figure 3. It can be seen that our model outperforms the baseline on questions requiring semantic knowledge and external knowledge. It seems that this result contradicts the above analysis, but it is reasonable. The baseline learns to answer all questions requiring different knowledge at the same layer, leading the model to tend to learn shortcuts (shortcuts are the tricks that use partial evidence to produce answers to the expected comprehension challenges, e.g., co-reference resolution) [47]. Our method exploits a multi-decision structure and a learning rate decaying method that makes the top block attend to learning semantic knowledge and external knowledge, which shows that our approach is effective on these questions. Figure 3: Examples requiring different information. Our model can explicitly exploit diverse information at different layers, and adaptively select the appropriate layer to answer the question. “\(\check{\vee}\)” indicates the selected layer.