Abstract: Event detection using social media streams needs a set of informative features with strong signals that need minimal preprocessing and are highly associated with events of interest. Identifying these informative features as keywords from Twitter is challenging, as people use informal language to express their thoughts and feelings. This informality includes acronyms, misspelled words, synonyms, transliteration and ambiguous terms. In this paper, we propose an efficient method to select the keywords frequently used in Twitter that are mostly associated with events of interest such as protests. The volume of these keywords is tracked in real time to identify the events of interest in a binary classification scheme. We use keywords within word-pairs to capture the context. The proposed method is to binarize vectors of daily counts for each word-pair by applying a spike detection temporal filter, then use the Jaccard metric to measure the similarity of the binary vector for each word-pair with the binary vector describing event occurrence. The top n word-pairs are used as features to classify any day to be an event or non-event day. The selected features are tested using multiple classifiers such as Naive Bayes, SVM, Logistic Regression, KNN and decision trees. They all produced AUC ROC scores up to 0.91 and F1 scores up to 0.79. The experiment is performed using the English language in multiple cities such as Melbourne, Sydney and Brisbane as well as the Indonesian language in Jakarta. The two experiments, comprising different languages and locations, yielded similar results.

Introduction: Event detection is important for emergency services to react rapidly and minimize damage. For example, terrorist attacks, protests, or bushfires may require the presence of ambulances, firefighters, and police as soon as possible to save people. This research aims to detect events as soon as they occur and are reported via some Twitter user. The event detection process requires to know the keywords associated with each event and to assess the minimal count of each word to decide confidently that an event has occurred. In this research, we propose a novel method of spike matching to identify keywords, and use probabilistic classification to assess the probability of having an event given the volume of each word. Event detection and prediction from social networks have been studied frequently in recent years. Most of the predictive frameworks use textual content such as likes, shares, and retweets, as features. The text is used as features either by tracking the temporal patterns of keywords, clustering words into topics, or by evaluating sentiment scores and polarity. The main challenge in keyword-based models is to determine which words to use in the first place, especially as people use words in a non-standard way, particularly on Twitter. In this research, we aim for detecting large events as soon as they happen with near-live sensitivity. For example, When spontaneous protests occur just after recent news such as increasing taxes or decreasing budget, we need to have indicators to raise the flag of a happening protest. Identifying these indicators requires to select a set of words that are mostly associated with the events of interest such as protests. We then track the volume of these words and evaluate the probability of an event occurring given the current volume of each of the tracked features. The main challenge is to find this set of features that allow such probabilistic classification. Using text as features in Twitter is challenging because of the informal nature of the tweets, the limited length of the tweet, platform-specific language, and multilingual nature of Twitter BIBREF0 , BIBREF1 , BIBREF2 . The main challenges for text analysis in Twitter are listed below: We approached the first and second challenges by using a Bayesian approach to learn which terms were associated with events, regardless of whether they are standard language, acronyms, or even a made-up word, so long as they match the events of interest. The third and fourth challenges are approached by using word-pairs, where we extract all the pairs of co-occurring words within each tweet. This allows us to recognize the context of the word ('Messi','strike' ) is different than ('labour','strike'). According to the distributional semantic hypothesis, event-related words are likely to be used on the day of an event more frequently than any normal day before or after the event. This will form a spike in the keyword count magnitude along the timeline as illustrated in Figure FIGREF6 . To find the words most associated with events, we search for the words that achieve the highest number of spikes matching the days of events. We use the Jaccard similarity metric as it values the spikes matching events and penalizes spikes with no event and penalizes events without spikes. Separate words can be noisy due to the misuse of the term by people, especially in big data environments. So, we rather used the word-pairs as textual features in order to capture the context of the word. For example, this can differentiate between the multiple usages of the word “strike” within the contexts of “lightning strike”, “football strike” and “labour strike” In this paper, we propose a method to find the best word-pairs to represent the events of interest. These word-pairs can be used for time series analysis to predict future events as indicated in Figure FIGREF1 . They can also be used as seeds for topic modelling, or to find related posts and word-pairs using dynamic query expansion. The proposed framework uses a temporal filter to identify the spikes within the word-pair signal to binarize the word-pair time series vector BIBREF3 . The binary vector of the word-pair is compared to the protest days vector using Jaccard similarity index BIBREF4 , BIBREF5 , where the word-pairs with highest similarity scores are the most associated word-pairs with protest days. This feature selection method is built upon the assumption that people discuss an event on the day of that event more than on any day before or after the event. This implies that word-pairs related to the event will form a spike on this specific day. Some of the spiking word-pairs are related to the nature of the event itself, such as “taxi protest” or “fair education”. These word-pairs will appear once or twice along the time frame. Meanwhile, more generic word-pairs such as “human rights” or “labour strike” will spike more frequently in the days of events regardless the protest nature. To test our method, we developed two experiments using all the tweets in Melbourne and Sydney over a period of 640 days. The total number of tweets exceeded 4 million tweets per day, with a total word-pair count of 12 million different word-pairs per day, forming 6 billion word-pairs over the entire timeframe. The selected word-pairs from in each city are used as features to classify if there will be an event or not on a specific day in that city. We classified events from the extracted word-pairs using 9 classifiers including Naive Bayes, Decision Trees, KNN, SVM, and logistic regression. In Section 2, we describe the event detection methods. Section 3 states the known statistical methods used for data association and feature selection. Section 4 describes the proposed feature selection method. Section 5 describes model training and prediction. Section 6 describes the experiment design, the data and the results. Section 7 summarizes the paper, discuss the research conclusion and explains future work.

Question: Which of the classifiers showed the best performance?
Answer: Logistic regression