# Cognitive Aspects-Based Short Text Representation with Named Entity, Concept and Knowledge
## 2 Related Work  
Short text classification is an important task of NLP. Many traditional methods like BoW, SVM and KNN, and so forth, have been explored for this task. In recent years, deep neural networks have been increasingly employed in the short text analysis. For example, [PERSON] [3] first introduced the Convolutional Neural Network (CNN for short) to the text classification. CNN is used to extract local and position-invariant features. Recurrent Neural Network (RNN for short) is another approach for the text processing. Unlike CNN, RNN is good at processing long range semantic dependency rather than local key-phrases. [PERSON] [11] proposed an attention model to process the problem of different words in a document with informative difference.  
The deep models aforementioned are flexible to some extent in the short text classification. However, due to the shortness and sparsity of short text, it is quite difficult for them to capture enough semantic information with limited words in the text content. From this perspective, how to enrich the short text semantic information with extra knowledge or common sense borrowed from other sources becomes a hot topic in this area. Concept is an aspect which is extensively used for text semantic enhancement. The Microsoft Concept Graph is a big graph of concepts, researchers have utilized it for the semantic enhancement. [PERSON] et al. [2] proposed a 'Bag-of-Concept' (instead of word) approach for the short text representation and constructs a concept model for each category, they then conceptualize the short text to a set of relevant concepts. [PERSON] et al. [7] proposed a deep convolutional neural network model, which utilizes the concept, word and character for short text classification. To measure the importance of each concept from the concept set, [PERSON] et al. [6] proposed a knowledge powered multiple attention networks for text classification, it applies two attention mechanisms to measure the importance of each concept from two aspects: the concept towards short text attention and the concept towards concept set attention.  
In addition, knowledge graph is another effective way to enhance the text semantic representation. A typical knowledge graph describes the structured and unstructured information with a Resource Description Framework (RDF). Information in the knowledge base is stored in the form of entity-relation-entity triples. There are many knowledge graph--DBpedia [12], Wikidata [13], Freebase [14] and YAGO [15]. They are widely employed in recent research on semantic enhancement for short text. [PERSON] et al. [16] devised a multi-channel CNN by fusing the word and knowledge graph levels of representations for news text representation. [PERSON] et al. [17] proposed a word and knowledge level-based self-attention mechanism for the text semantic enhancement.  
For further semantic enhancement, entity is usually utilized together with the knowledge base. [PERSON] et.al [5] proposed an entity-based text classification, it utilizes entity and its related attributes for the short text enhancement. [PERSON] [18] proposed a knowledge-based short text categorization, which utilizes the external knowledge base (Wikipedia) and entity.