Question: What will be acquired in the output representation of the first stage?
Evidence: We propose an attention convolutional matching layer to integrate two similarity maps given above. That is, QACNN Layer is used to learn the location relationship pattern. It contains a two-staged CNN combined with query-based attention mechanism. Each stage comprises two major part: attention map and output representation.
Fig. FIGREF10 shows the architecture of the attention map in first stage CNN. We choose INLINEFORM0 sentence slice INLINEFORM1 in INLINEFORM2 , and apply CNN to it using the convolution kernel INLINEFORM3 , where superscript INLINEFORM4 denotes attention map, subscript INLINEFORM5 denotes the first stage CNN. Symbol INLINEFORM6 and INLINEFORM7 represent width of kernel and number of kernel respectively. The generated feature INLINEFORM8 is as follow: DISPLAYFORM0 
where INLINEFORM0 is the bias. With INLINEFORM1 covering whole query and several words in the passage, convolution kernels would learn the query syntactic structure and give weight to each passage's location. That's why we use sigmoid function as activation function in this stage. Furthermore, we perform maxpooling to INLINEFORM2 perpendicularly in order to find the largest weight between different kernels in the same location, using maxpool kernel shaped INLINEFORM3 , and then generate word-level attention map INLINEFORM4 for each sentence.
In this stage, we want to acquire passage's sentence features based on the query and the choice respectively. We apply CNN to INLINEFORM0 to aggregate pattern of location relationship and acquire choice-based sentence features. Also, we apply CNN to INLINEFORM1 to acquire query-based sentence features. CNN architecture of output representation part Fig. FIGREF11 is similar to which of attention map part, but we use different kernels INLINEFORM2 and different bias INLINEFORM3 : DISPLAYFORM0 
where the superscript INLINEFORM0 denotes output representation. We apply INLINEFORM3 on INLINEFORM4 and INLINEFORM5 then finally generate INLINEFORM6 and INLINEFORM7 using eq.4. We then multiply INLINEFORM8 by the word-level attention map INLINEFORM9 which comes from stage 2.2.1 element-wise through the first dimension. At last, we maxpool INLINEFORM10 and INLINEFORM11 horizontally with kernel shape INLINEFORM12 to get the query-based sentence features INLINEFORM13 and choice-based sentence features INLINEFORM14 .
Fig. FIGREF28 is the architecture of attention map in the second stage CNN. Based on first stage query-based sentence features from section UID15 , we want to acquire sentence-level attention map. The input of this stage is INLINEFORM0 , which will be further refined by CNN with kernel INLINEFORM1 and generates intermediate features INLINEFORM2 . DISPLAYFORM0 
Then, same as attention map of first stage, we maxpool INLINEFORM0 with kernel shaped INLINEFORM1 , and obtain sentence-level attention map INLINEFORM2 .
Output representation part of the second stage in Fig. FIGREF29 has two input, sentence-level attention map INLINEFORM0 and sentence-level features INLINEFORM1 . The equations here are similar to those previously mentioned. As follow: DISPLAYFORM0 
where INLINEFORM0 , INLINEFORM1 , and INLINEFORM2 . Output representation of certain choice INLINEFORM3 is the final output of QACNN Layer.
Output Representation of First Stage In this stage, we want to acquire passage’s sentence features based on the query and the choice respectively.
Answer: Passage' s sentence features based on the query and the choice respectively.