# Chinese Neural Question Generation: Augmenting Knowledge into Multiple Neural Encoders
## 2 Related Work  
### Paragraph-Level Question Generation  
Utilizing paragraph-level contexts around the input text is an important factor to generate good questions. [PERSON] et al. [12] observed that 20% of questions in the SQuAD dataset require paragraph-level information to be answered. However, as the input texts get longer, it becomes more challenging for sequence-to-sequence neural models to learn to\" copy\" relevant context and avoid irrelevant information. To overcome the challenges of processing long sequences of text inputs, [PERSON] et al. [16] introduced a gated self-attention encoder with a maxout pointer mechanism. The gated self-attention encoder first takes an encoded passage-answer representation as input and performs matching against itself to compute a self-matching representation. Then, the encoder combines the input with the self-matching representation using a feature fusion gate. The decoder used a maxout pointer mechanism [21] to overcome the repetition problem (the repeated occurrence of words in the input text tends to cause repetitions in output sequence). The study results showed that the gated self-attention mechanism helped the decoder focus on the context words surrounding the answer, and thus reached good performance in question generation. Nonetheless, we argue, if the context words used for human question generation are far from the answer, the model may fail to generate questions \"as natural\" as humans generate. To capture richer contextual representation of passage, [PERSON] and [PERSON] [19] used the Bidirectional Encoder Representations from Transformers (BERT) to encode the passage. The decoder only contains a fully connected layer connecting from the output of the last token hidden state in the input sequence. They found that this BERT passage encoder outperformed [PERSON]'s question generation model on English SQuAD dataset.  
Instead of using the whole paragraph as context information for question generation, [PERSON] and [PERSON] [15] considered only relevant sentences, i.e., sentences containing the answer. In addition, co- reference information in the sentences served as an extra feature into the encoder. The co-reference information is obtained by a gate mechanism [22]. The study results showed that the co-reference representation improved their question generation system significantly. However, in Chinese the anaphora is often missing, called zero pronouns, which brings challenges with respect to explicitly capturing their semantic similarities with antecedents [17]. This will cause a major challenge with obtaining correct co-reference information in Chinese para-graphs. Inspired by [PERSON] and [PERSON]'s work, we introduce a passage compression step, which only selects relevant sentences containing the answer. However, this process has its own challenge: it may lose useful information about the answer. Therefore, we added information from a knowledge graph to compensate for the loss of irrelevant sentences. A knowledge graph or base is a structured database, which contains entities and relationships between those entities usually in the form of a triple (entity1, relation, entity2). In recent years, several large Chinese encyclopedia knowledge graphs were made publicly available on cloud, such as CN-DBpedia2 [23]. For example, CN-DBpedia2 [23] contains around 16,024,656 entities and 228,499,155 relation types. It was developed based on Baidu Baike [24], Hudong Baike [25], Chinese Wikipedia [26] and other encyclopedia websites. Recently, deep neural networks have been used to generate factual questions based on triples in the CN-DBpedia2 knowledge graph [27]. In the current study, we extracted and ranked the triples related to the answer entity and which are semantically similar to the filtered sentences. Thus, the filtered information about the answer is largely retained and to some degree expanded.