Question: How many SQA models (QANets) does the proposed approach have?
Evidence: The main focus of this paper is to apply domain adaptation for SQA. In this approach, we have two SQA models (QANets), one trained from target domain data (ASR hypotheses) and another trained from source domain data (reference transcriptions). Because the two domains share common information, some layers in these two models can be tied in order to model the shared features. Hence, we can choose whether each layer in the QA model should be shared. Tying the weights between the source layer and the target layer in order to learn a symmetric mapping is to project both source and target domain data to a shared common space. Different combinations will be investigated in our experiments.
More specifically, we incorporate a domain discriminator into the SQA model shown in Figure FIGREF2 , which can enforce the embedding encoder to project the sentences from both source and target domains into a shared common space and consequentially to be ASR-error robust. Although the embedding encoder for both domains may implicitly learn some common latent representations, adversarial learning can provide a more direct training signal for aligning the output distribution of the embedding encoder from both domains. The embedding encoder takes in a sequence of word vectors and generates a sequence of hidden vectors with the same length. We use INLINEFORM0 and INLINEFORM1 ( INLINEFORM2 and INLINEFORM3 ) to represent the hidden vector sequence given the question INLINEFORM4 and the document INLINEFORM5 in the target (source) domain respectively.
The domain discriminator INLINEFORM0 focuses on identifying the domain of the vector sequence is from given INLINEFORM1 or INLINEFORM2 , where the objective is to minimize INLINEFORM3 . DISPLAYFORM0 
Given a training example from the target domain ( INLINEFORM0 ), INLINEFORM1 learns to assign a lower score to INLINEFORM2 and INLINEFORM3 in that example, that is, to minimize INLINEFORM4 and INLINEFORM5 . On the other hand, given a training example from the source domain ( INLINEFORM6 ), INLINEFORM7 learns to assign a larger value to INLINEFORM8 and INLINEFORM9 .
Furthermore, we update the parameters of the embedding encoders to maximize the domain classification loss INLINEFORM0 , which works adversarially towards the domain discriminator. We thus expect the model to learn features and structures that can generalize across domains when the outputs of INLINEFORM1 are indistinguishable from the outputs of INLINEFORM2 . The loss function for embedding encoder, INLINEFORM3 , is formulated as DISPLAYFORM0 
where INLINEFORM0 is a hyperparameter. The two embedding encoders in the QA model are learned to maximize INLINEFORM1 while minimizing the loss for QA, INLINEFORM2 . Because the parameters of other layers in QA model are independent to the loss of the domain discriminator, the loss function of other layers, INLINEFORM3 , is equivalent to INLINEFORM4 , that is, INLINEFORM5 .
Although the discriminator is applied to the output of embedding encoder in Figure FIGREF2 , it can be also applied to other layers. Considering that almost all QA model contains such embedding encoders, the proposed approach is expected to generalize to other QA models in addition to QANet.
In this approach, we have two SQA models (QANets), one trained from target domain data (ASR hypotheses) and another trained from source domain data (reference transcrip tions).
Answer: Two SQA models (QANets).