Question: What can be done to reduce sparsity?
Evidence: Question classification typically makes use of a combination of syntactic, semantic, surface, and embedding methods. Syntactic patterns BIBREF18 , BIBREF19 , BIBREF20 , BIBREF21 and syntactic dependencies BIBREF3 have been shown to improve performance, while syntactically or semantically important words are often expanding using Wordnet hypernyms or Unified Medical Language System categories (for the medical domain) to help mitigate sparsity BIBREF22 , BIBREF23 , BIBREF24 . Keyword identification helps identify specific terms useful for classification BIBREF25 , BIBREF3 , BIBREF26 . Similarly, named entity recognizers BIBREF6 , BIBREF27 or lists of semantically related words BIBREF6 , BIBREF24 can also be used to establish broad topics or entity categories and mitigate sparsity, as can word embeddings BIBREF28 , BIBREF29 . Here, we empirically demonstrate many of these existing methods do not transfer to the science domain.
The highest performing question classification systems tend to make use of customized rule-based pattern matching BIBREF30 , BIBREF7 , or a combination of rule-based and machine learning approaches BIBREF19 , at the expense of increased model construction time. A recent emphasis on learned methods has shown a large set of CNN BIBREF29 and LSTM BIBREF8 variants achieve similar accuracy on TREC question classification, with these models exhibiting at best small gains over simple term frequency models. These recent developments echo the observations of Roberts et al. BIBREF3 , who showed that existing methods beyond term frequency models failed to generalize to medical domain questions. Here we show that strong performance across multiple datasets is possible using a single learned model.
Due to the cost involved in their construction, question classification datasets and classification taxonomies tend to be small, which can create methodological challenges. Roberts et al. BIBREF3 generated the next-largest dataset from TREC, containing 2,936 consumer health questions classified into 13 question categories. More recently, Wasim et al. BIBREF4 generated a small corpus of 780 biomedical domain questions organized into 88 categories. In the education domain, Godea et al. BIBREF5 collected a set of 1,155 classroom questions and organized these into 16 categories. To enable a detailed study of science domain question classification, here we construct a large-scale challenge dataset that exceeds the size and classification specificity of other datasets, in many cases by nearly an order of magnitude.
Syntactic patterns (Li and Roth, 2006; Silva et al., 2011; Patrick and Li, 2012; Mishra et al., 2013) and syntactic dependencies (Roberts et al., 2014) have been shown to improve performance, while syntactically or semantically important words are often expanding using Wordnet hypernyms or Unified Medical Language System categories (for the medical domain) to help mitigate sparsity (Huang et al., 2008; Yu and Cao, 2008; Van-Tu and Anh-Cuong, 2016). Keyword identification helps identify specific terms useful for classification (Liu et al., 2011; Roberts et al., 2014; Khashabi et al., 2017). Similarly, named entity recognizers (Li and Roth, 2002; Neves and Kraus, 2016) or lists of semantically related words (Li and Roth, 2002; Van-Tu and Anh-Cuong, 2016) can also be used to establish broad topics or entity categories and mitigate sparsity, as can word embeddings (Kim, 2014; Lei et al., 2018).
Answer: Syntactic patterns and syntactic dependencies have been shown to improve performance, while syntactically or semantically important words are often expanding using Wordnet hypernyms or Unified Medical Language System categories to help mitigate sparsity. Keyword identification helps identify specific terms useful for classification. Similarly, named entity recognizers or lists of semantically related words can also be used to establish broad topics or entity categories and mitigate sparsity, as can word embeddings.