# Chinese Medical Question Answer Matching Using End-to-End Character-Level Multi-Scale CNNs
## 3 Methodology
### Distributed Representation of Characters  
In many natural language processing tasks, one fundamental step is to convert text sequences to machine-readable features, which are always fixed-length vectors. These features determine the upper limit of the effectiveness of the algorithms. In recent years, embedding-based methods have been used to extract features from text, which shows their utility in semantic representation. A useful and often-mentioned type of embedding method is word embedding, also referred to as the distributed word representation. [PERSON] et al. [20] proposed a neural network language model (NNLM) which combined a neural network with natural language processing to train word embeddings. After that, [PERSON] et al. [21] proposed Word2 Vec, a very effective language model inspired from NNLM. Word2 Vec has received increasing attention in recent years and has been successfully applied to many tasks, such as document classification [22; 23], knowledge graph extraction [24; 25], and implicit matrix factorization [26; 27]. Shown in Figure 2a is an example of the Word2 Vec model. Given a sentence sequence \(Sent=[w_{1},w_{2},\cdots,w_{l_{w}}]\), where \(l_{w}\) is the length of the sequence and \(w_{i}\in\mathbb{R}^{d_{w}}\) indicates the word vector of the \(i^{th}\) word in the sentence. Let \(Context(w_{i})=[w_{i-k}\cdots,w_{i-1},w_{i+1},\cdots,w_{i+k}]\) be the context of \(w_{i}\), where \(2k\) is the size of the contextual window. Let \(p(w_{i}|Context(w_{i}))\) be the probability of the \(i^{th}\) word in the sentence being \(w_{i}\). The target of the model is to optimize the log maximum likelihood estimation (logMLE):  
\[\mathcal{L}_{w}(\text{MLE})=\sum_{w_{i}\in s}\log p(w_{i}|Context(w_{i})). \tag{1}\]  
However, word-level embedding methods may suffer from the following shortcomings:  
1. Word segmentation is an essential preprocessing step for most Chinese NLP tasks. The quality of word segmentation determines the quality of word embeddings, which then influence the accuracy of the down-stream model. Moreover, word segmentation methods suffer a great reduction in accuracy when directly applied to specific domains (e.g., medical domain). Figure 1: The framework of our model. The question and the answer are first split into characters, and they are represented by character embeddings. The embeddings are then fed into a multi-scale convolutional neural network architecture to extract local information. After that, max pooling layer reduces the representation by choosing the max value along the sequence. Finally, the vectors are concatenated to calculate the similarity of the question and the answer. It should be noted here that layers with the same color share parameters and weights. 2. The words that do not or rarely appear in the lexicon or dictionary are called unseen words or rare words. Unseen or rare words may influence the quality of word embeddings. 3. As the number of words is far greater than that of characters, training word embeddings or adopting word embeddings as features may consume a great deal of computing resources and storage resources. Inspired by [PERSON] et al. [28], we separate the sentence into individual characters. In order to train distributed representation of characters, we have made minor changes to Word2 Vec. As can be seen from Figure 2b, in the context window, each character is used to predict the middle character. After being trained, each character is mapped into a fixed-length vector \(c_{i}\in\mathbb{R}^{d_{c}}\), where \(d_{c}\) is the dimensionality. More specifically, given a sentence sequence \(Sent=[c_{1},c_{2},\cdots,c_{l_{c}}]\), where \(l_{c}\) is the number of characters in the sequence, the context of \(c_{i}\) is \(Context(c_{i})=[c_{i-k},\cdots,c_{i-1},c_{i+1},\cdots,c_{i+k}]\). Therefore, Equation (1) can be modified into:  
\[\mathcal{L}_{c}(\text{MLE})=\sum_{c_{i}\in s}\log p(c_{i}|Context(c_{i})). \tag{2}\]  
Character embeddings reduce the chances of errors brought about by word segmentation algorithms [28].