# Local Event Detection Scheme by Analyzing Relevant Documents in Social Networks
## 4 Experiments  
We verified the superiority of the proposed scheme by comparing its performances with the existing event detection schemes. The experimental environment is shown in Table 1. We conducted various experiments on single server environments, where the server was equipped with an Intel core i5-3570 CPU 3.40 GHz and 16 GB of memory. We implemented the proposed scheme by using Python 2.7 on Window OS. We collected 119,243 tweets and 137,942 relevant documents by using Twitter's standard API. We collected 1 month of tweets and documents (from 1 August 2020). The proposed scheme collected only specific topic-based data. In other words, since the aim of the proposed scheme is to detect events occurring in the real world, tweets were collected based on event words such as fires, typhoons, COVID-19, car accidents, earthquakes, floods, and demonstrations that could occur offline. In this paper, the ground truth was annotated by human users. In the experiments, we used 100 local events as the ground truth. The ground truth is called the true set in the paper. In information retrieval, recall is the fraction of the relevant documents that are successfully retrieved [34]. In addition, precision is the fraction of retrieved documents that are relevant to the query on the event. For example, for an event search on a set of documents, precision is the number of correct results divided by the number of all returned results. F-measure is a measure that combines precision and recall; it is the harmonic mean of precision and recall. Therefore, it is calculated as follows:  
\[F_{measure}=2*\frac{precision*recall}{precision+recall} \tag{6}\]The recall was calculated by changing \(k\) from 5 to 20 (maximum/20 to max/5). Recall calculation values are rounded; \(k\) is the value that represents the number of events that each event detection scheme extracts. As we changed k from 5 to 20, we performed a comparison between the search results of the proposed technique and the set of correct answers. Accuracy is calculated by assigning a value of 0.5 if the clustered event is the same as the true set, and 0.5 if the keywords that are considered to have the same meaning as the tagged event category are clustered. The existing schemes such as text-based event detection [6] and geo-tag based event detection [17] were chosen for comparison. We compared precision, recall, and F-measure to show the excellence of the proposed scheme. The proposed scheme, the text-based event detection [6], and geo-tag based event detection [17] are denoted as T+R, TBEM, and GTBEM, respectively. The denotation OnlyTweet refers to using the event detection scheme without considering relevant documents. The results of the clustering depend on a threshold. Therefore, we derived an optimized value through experiments. The proposed technique considers one community (cluster) as one event in the graph. \(NM_{G}\) is a quantitative representation of how well each cluster is separated. The higher the value is, the better the separation of events is. Therefore, we find the highest \(NM_{G}\) value while changing the threshold value and utilize the threshold at the highest \(NM_{G}\) for clustering. Figure 7 shows a comparison of \(NM_{G}\) according to different thresholds. In the experiments, thresholds were assigned a value from 0.1 to 1. When the threshold is 0.1, we cannot extract important local events since the redundant keywords were contained in the event cluster. Since the threshold shows the best \(NM_{G}\) over the other \(NM_{G}\) when the threshold is 0.5, we set the threshold to 0.5. Figure 8 shows a comparison of precision according to the local event detection schemes. GTBEM and TBEM show relatively low performance (about 40%), as it is hard to derive regional information because they use only a text mining algorithm and geo-tags, respectively. GTBEM shows the lowest performance (about 38%) since it only considered geo-tags. OnlyTweet, which does not consider relevant documents, ranked second (about 65%).