# A Review on Medical Textual Question Answering Systems Based on Deep Learning Approaches
## 3 Deep Learning Based MQA Approaches
### Convolutional Neural Networks Based Models  
The use of convolutional neural networks (CNNs) are inspired by the processes in the visual cortex of animals [60]. Each neuron that constitutes the visual cortex is covered with a receptive field (i.e., region under the filter) of the image. These receptive fields overlap with the whole image to allow its complete visualization. Typically, CNNs have three main types of layer: an input layer, a feature extraction layer and a classification layer. CNN models are extremely effective in feature representation, including object recognition [67], sentiment analysis [14], and question answering [34]. A CNN architecture typically consists of two main processes: convolution and pooling. The convolution process is responsible for extracting features from the input content using sliding filters. On the other hand, the pooling process selects the maximum or average value of the features extracted from the former process (i.e., convolution) to reduce the feature map size. In the QA system, the question and answer are represented by character embedding sequences, which can be denoted by \(c_{1},\dots,c_{l_{c}}\), and \(c_{i}\in\mathbb{R}^{d_{c}}\), where \(d_{c}\). is the dimensionality of the character vectors. Each sentence is normalized to obtain a fixed-length sequence. After embedding, each question and answer can be represented by matrix \(Q_{e}\in\mathbb{R}^{l_{c}\times d_{c}}\) and \(A_{e}\in\mathbb{R}^{l_{c}\times d_{c}}\). Given a sequence, \(Z=[z_{1},z_{2},\dots,z_{l_{1}}-s+1]\), where \(s\) is the size of the feature map, and \(z_{i}=[c_{i},c_{i+1},\dots,c_{i+s-1}]\in\mathbb{R}^{s\times d_{c}}\) is the concatenation of continuous \(s\) character vectors of the sentence. The convolutional operation can be defined as shown in Equation (4)  
\[O_{j}=f\big{(}W_{j}\circ[z_{1},z_{2},\dots,z_{l_{c}-s+1}]+b\big{)} \tag{4}\]  
where \(O_{j}\in\mathbb{R}^{l_{c}-s+1}\) is the output of the convolutional layer, \(W_{j}\in\mathbb{R}^{s\times d}\) and \(b\) are the parameters to be trained, \(W\circ Z\) indicates the element-wise multiplication of \(W\) with each element in \(Z\), and \(f(\cdot)\) is the activation function. Through the convolutional layer, \(Q_{E}\in\mathbb{R}^{l_{c}\times d_{c}}\) and \(A_{E}\in\mathbb{R}^{l_{c}\times d_{c}}\) can be converted into \(Q_{O}\in\mathbb{R}^{(l_{c}-s+1)\times d_{c}}\) and \(A_{O}\in\mathbb{R}^{(l_{c}-s+1)\times d_{c}}\), where \(d_{c}\) is the number of filter maps. Then, a pooling layer is used after the convolutional layer. The pooling layer chooses the max or average value of the features extracted from the former layer, which can reduce the representation. Equation (5) explains how the max-pooling operation is performed  
\[p=[\max O_{1},\max O_{2},\dots,\max O_{d_{o}}] \tag{5}\]  
where \(\max O_{i}\) is the max value of \(O_{i}\) and \(p\in\mathbb{R}^{d_{c}}\) is the output of the pooling layer. Equation (6) shows how to measure the similarity between the questions and the answers. \[\text{Sim}(q,a)=\text{Cos}(q,a)=\frac{\|q\cdot a\|}{\|q\|\cdot\|a\|^{\prime}} \tag{6}\]  
where \(\|\cdot\|\) is the length of the vector, \(q\in\mathbb{R}^{d_{o}}\) and \(a\in\mathbb{R}^{d_{o}}\) are the output of the max-pooling layer, used to represent the question and answer, respectively. In [60], [PERSON], et al. proposed the multi-scale convolutional neural network's (multi-CNNs) architecture in [60]. This end-to-end character-level architecture was employed to deal with the Chinese MQA matching task, as shown in Figure 9. The authors in [60] introduced an architecture for extracting contextual information from either question or answer sentences over various scales. Both questions and answers are all limited to the Chinese language. The reason for their choice of character embedding over word embedding is to avoid the segmentation of Chinese word in text preprocessing.