An Online Semantic-enhanced D irichlet Model for Short Text Stream Clustering (2020)

Abstract: Clustering short text streams is a challenging task due to its unique properties: infinite length, sparse data representation and cluster evolution. Existing approaches often exploit short text streams in a batch way. However, determine the optimal batch size is usually a difficult task since we have no priori knowledge when the topics evolve. In addition, traditional independent word representation in graphical model tends to cause “term ambiguity” problem in short text clustering. Therefore, in this paper, we propose an Online Semantic-enhanced Dirichlet Model for short sext stream clustering, called OSDM, which integrates the word-occurance semantic information (i.e., context) into a new graphical model and clusters each arriving short text automatically in an online way. Extensive results have demonstrated that OSDM has better performance compared to many state-of-the-art algorithms on both synthetic and real-world data sets.

## introduction a massive amount of short text data is constantly generated with online social platforms such as microblogs, twitter and facebook. clustering of such short text streams has thus gained increasing attention in recent years due to many real-world applications like event tracking, hot topic detection, and news recommendation @xcite . however, due to the unique properties of short text streams such as infinite length, evolving patterns and sparse data representation, short text stream clustering is still a big challenge @xcite . during the past decade, many approaches have been proposed to address the text stream clustering problem from different points of view, and each method comes with specific advantages and drawbacks. initially, traditional clustering algorithms for static data were enhanced and transformed for text streams @xcite . very soon, they are replaced by model-based algorithms such as lda @xcite , dtm @xcite , tdpm @xcite , gsdmm @xcite , dpmfp @xcite , tm-lda @xcite , npmm @xcite and mstream @xcite , to mention a few. however, for most established approaches, they often work in a batch way, and assume the instances within a batch are interchangeable. this assumption usually cannot hold for topic-evolving text data corpus. determining an optimal batch size is also a non-trivial task for different text streams @xcite . additionally, unlike long text documents, short text clustering further suffers from the lack of supportive term occurrence to capture semantics @xcite . for most existing short text clustering algorithms like sumblr @xcite , dct @xcite and mstreamf @xcite , exploiting independent word representation in their cluster models tends to cause ambiguity. let us show the following four tweets, for example: t1: "a regular intake of an apple can improve your health and muscle stamina." t1: "a glass of fresh apple juice is recommended for breakfast." t2: "new apple watch can monitor your health." t2: "apple will launch new smartphone iphonex this december." tweets of these two topics share few common terms, i.e., 'health' or 'apple'. it creates an ambiguity if the model deals with only single term representation to calculate the similarity. however, the co-occurring terms representation (i.e., context) helps a model to identify the topic 1 correctly. to solve these aforementioned issues, we propose an online semantic-enhanced dirichlet model for short text stream clustering. compared to existing approaches, it has following advantages. (1) it allows processing each arriving short text in an online way. the online model is not only free of determining the optimal batch size, but also lends itself to handling large-scale data streams efficiently; ## related work during the past decade, many text stream clustering algorithms have been proposed. here, due to the space limitation, we only report some model-based approaches which are highly related to our work. for more details, please refer to comprehensive surveys, e.g., @xcite @xcite . the early classical attempt for text clustering is latent dirichlet allocation (lda) @xcite . however, it cannot handle the temporal data for text streams. for this purpose, many lda variants have been proposed to consider the text streams such as dynamic topic model (dtm) @xcite , dynamic mixture model (dmm) @xcite , temporal lda (t-lda) @xcite , streaming lda (s-lda) @xcite , and dirichlet mixture model with feature partition (dpmfp) @xcite . these models assume that each document contains rich content, and thus they are not suitable for dealing with the short text streams. later, dirichlet multinomial mixture model-based dynamic clustering topic (dct) model was designed to deal with short text streams by assigning each 1 topic and cluster will be interchangeably used in this paper document with single topic @xcite . very soon, gsdmm was proposed to extend dmm with collapsed gibbs sampling to infer the number of clusters @xcite . however, most of these models did not investigate the evolving topics (clusters) in text streams where the number of topics usually evolves over time. to automatically detecting the number of clusters, @xcite proposed a temporal dirichlet process mixture model (tdmp). it divides the text stream into many chunks (batches), and assumes that the documents inside each batch are interchangeable. later, gsdpmm was proposed with collapsed gibbs sampling to infer the number of clusters in each batch. in contrast to lda, gsdpmm not only converges faster but also dynamically assigns the number of clusters over time @xcite . however, both tdmp and gsdpmm models do not examine the evolving topics, and, these models process the text stream for multiple times. thereafter, mstreamf @xcite was thus proposed by incorporating a forgetting mechanism to cope with cluster evolution, and allows processing each batch only one time. the npmm model @xcite was recently introduced by using the word-embeddings to eliminate a cluster generating parameter of the model. in summary, for most existing approaches, they usually work in a batch way. however, determining optimal batch sizes for different text streams is usually a difficult task. more importantly, due to the intrinsic sparse data representation of shorttext data, the semantics, is little investigated in established approaches. actually, they need to be carefully considered to decrease the term ambiguity in short text clustering. ## preliminaries here, the problem statement is first given, followed with a brief introduction about dirichlet process and poly urn scheme. ## proposed approach this section gives a brief discussion about the representation and formulation of the proposed algorithm. ## conclusion in this paper, we propose a new online semanticenhanced dirichlet model for short text stream clustering. in contrast to existing approaches, osdm does not require to specify the batch size and the dynamic number evolving clusters. it dynamically assigns each arriving document into an existing cluster or generating a new cluster based on the poly urn scheme. more importantly, osdm tried to incorporate semantic information in the proposed graphical representation model to remove the term ambiguity problem in short-text clustering. building upon the semantic embedding and online learning, our method allows finding high-quality evolving clusters. extensive results further demonstrate that osdm has better performance compared to many state-of-the-art algorithms.