# A Nested Chinese Restaurant Topic Model for Short Texts with Document Embeddings
## 1 Introduction  
With the growth of social media and applications of mobile phones, short texts have been a kind of prevalent and important information on the internet. There is abundant semantic information that can be found in short texts. However, unlike documents with regular size, the average length of each short text is very short. This characteristic makes the knowledge discovery on short texts being a challenging research problem. To discover knowledge from texts, the topic modeling method is widely used [1]. For documents with regular size, conventional topic models like LDA [2] and HDP [3] perform well. These methods can automatically generate topics according to word co-occurrence information. However, for short texts, word co-occurrence information is very sparse because of the short length of each text [4]. So the performances of these conventional methods are very poor on short texts [5]. To overcome this problem, many researchers propose different kinds of customized topic models dealing with the sparsity of word co-occurrence information [6]. As word co-occurrence information in short texts is sparse, researchers propose two kinds of strategies to overcome this problem. The first strategy is to incorporate auxiliary information. Metadata like hashtags or authors is available in short texts. Some methods aggregate short texts according to metadata [7, 8, 9]. However, these methods are effective only for a few short texts corpus because metadata is not always available. So other methods incorporate word embeddings generated from an auxiliary corpus with documentsof regular length [10; 11; 12; 13; 14]. However, these methods need to construct a semantically corresponding auxiliary corpus of short texts. If the auxiliary corpus is not semantically related to target short texts, the performance will be very poor [15]. The second strategy is to design specific models without auxiliary information. Several models restrict the number of topics each short text corresponds to [16; 17; 18]. The local word co-occurrence information of each topic is very sparse. So when a short text corresponds to one or a few topics, the local word co-occurrence information will be less sparse. However, this strategy cannot add sufficient word co-occurrence information. Several models use the global word co-occurrence information [19; 20]. These methods rebuild short texts corpus and construct a global word pairs set. The word pairs set is the summary of local word co-occurrence information. So the global word co-occurrence information will be less sparse. However, in these models, variables of documents are missing and the probability distribution between short texts and topics is unable to be obtained directly. Self-aggregated topic models [21; 22] automatically aggregate short texts into latent long documents. Then topics are generated from these long documents. Long documents incorporate sufficient word co-occurrence information and make local word co-occurrence information no longer sparse. These models seem more reasonable than other strategies and auxiliary information is not needed. However, the aggregation process of short texts may incorporate non-semantic information. Short texts that are not semantically relevant may be aggregated into one long document. Then this long document will bring a lot of non-semantic word co-occurrence information into topics and makes topics incoherent. As there exist a lot of deficiencies of the state-of-the-art topic models, we propose a self-aggregated topic model by incorporating document embeddings [23] (DESTM) to deal with the sparsity of word co-occurrence information. Document embedding information of short texts is the mapping of short texts' content in vector space. This information contains two kinds of information: word co-occurrence information and context semantic information. Then we can get the similarities between every two short texts according to document embeddings.