Event Detection in Twitter ∗

 Event Detection in Twitter Jianshu Weng; Yuxia Yao; Erwin Leonardi; Francis Lee HP Laboratories HPL-2011-98 twitter; event detection; wavelet Twitter, as a form of social media, is fast emerging in recent years. Users are using Twitter to report real-life events. This paper focuses on detecting those events by analyzing the text stream in Twitter. Although event detection has long been a research topic, the characteristics of Twitter make it a non-trivial task. Tweets reporting such events are usually overwhelmed by high flood of meaningless "babbles". Moreover, event detection algorithm needs to be scalable given the sheer amount of tweets. This paper attempts to tackle these challenges with EDCoW (Event Detection with Clustering of Wavelet-based Signals). EDCoW builds signals for individual words by applying wavelet analysis on the frequency-based raw signals of the words. It then filters away the trivial words by looking at their corresponding signal auto-correlations. The remaining words are then clustered to form events with a modularity-based graph partitioning technique. Experimental studies show promising result of EDCoW. We also present the design of a proof-of-concept system, which was used to analyze netizens’ online discussion about Singapore General Election 2011. External Posting Date: July 06, 2011 [Fulltext] Approved for External Publication Internal Posting Date: July 06, 2011 [Fulltext] Copyright 2011 Hewlett-Packard Development Company, L.P. Event Detection inTwitter Jianshu Weng, Yuxia Yao, Erwin Leonardi, and Bu-Sung Lee Services Platform Lab, HP Labs Singapore {jianshu.weng,yuxia.yao,erwin.leonardi,francis.lee }@hp.com Abstract Twitter, as a form of social media, is fast emerging in recent years. U sers are usingTwitter to report real-life events. This paper focuses on detecting those events by analyzing the text stream in Twitter. Althoughevent detectionhas long been a research topic, the characteristics of Twitter make it a non-trivial task. Tweets reporting such events are usually overwhelmed by high flood of m eaningless “babbles”. Moreover, event detection algorithm needs to be scal able given the sheer amount of tweets. This paper attempts to tackle these challe nges withEDCoW (Event Detection withClusteringof Wavelet-based Signals). EDCoW builds signals for individual words by applying wavelet analysis o n the frequency-based raw signals of the words. It then filters away the trivial word s by looking at their corresponding signal auto-correlations. The remaining wo rds are then clustered to form events with a modularity-based graph partitioning tec hnique. Experimental studies show promising result of EDCoW. We also present the d esign of a proofof-concept system, which was used to analyze netizens’ onli ne d scussion about Singapore General Election 2011.Twitter, as a form of social media, is fast emerging in recent years. U sers are usingTwitter to report real-life events. This paper focuses on detecting those events by analyzing the text stream in Twitter. Althoughevent detectionhas long been a research topic, the characteristics of Twitter make it a non-trivial task. Tweets reporting such events are usually overwhelmed by high flood of m eaningless “babbles”. Moreover, event detection algorithm needs to be scal able given the sheer amount of tweets. This paper attempts to tackle these challe nges withEDCoW (Event Detection withClusteringof Wavelet-based Signals). EDCoW builds signals for individual words by applying wavelet analysis o n the frequency-based raw signals of the words. It then filters away the trivial word s by looking at their corresponding signal auto-correlations. The remaining wo rds are then clustered to form events with a modularity-based graph partitioning tec hnique. Experimental studies show promising result of EDCoW. We also present the d esign of a proofof-concept system, which was used to analyze netizens’ onli ne d scussion about Singapore General Election 2011.